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ABSTRACT 



This volume summarizes projects funded by the Fund for the 
Improvement of Postsecondary Education, examiniiLg the various lessons 
learned. The 39 reports are organized in nine sections titled: (1) "Preface"; 

(2) "Access and Retention"; (3) Improving Teaching and Learning"; (4) 
"Curriculum and Instruction"; (5) "General Education"; (6) "International 
Education"; (7) "Teacher Education"; (8) '’Performance Funding and Reporting 
in Colleges and Universities"; and (9) "Disseminating Proven Reforms." Each 
project report describes its purpose, innovative features, evaluation and 
project impact, lessons learned, project continuation and dissemination, and 
contacts for further information. Overall, most projects were deemed 
successful. Lessons gleaned from the projects include the fact that holding 
students to high standards works; passionate, creative teaching is crucial; 
difficulties will arise and can be surmounted; and excellent work eventually 
brings recognition and reward. (SM) 
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Preface 

Lessons from Lessons Learned 



Even as FIPSE strives to bring about reform in postsecondary education, its own fund of 
knowledge and experience is constantly shaped and enhanced by the projects it supports. 

A volume such as this one, which summarizes some of the best projects that we have funded in 
recent years, is as rich in lessons for FIPSE as we hope it will be for the field. It may be 
useful by way of introduction to identify some of these lessons. Some of them we at FIPSE 
have long known to be true. Others are as surprising to us as they were to the projects’ 
creators. All will, we hope, prove helpful and encouraging to those who care about education. 

When first conceiving a project, it is prudent to investigate first-hand the causes of the 
problem to be addressed. For example, the high college attrition of General Educational 
Development (GED) certificate holders is commonly attributed to the same lack of motivation 
that kept them from finishing high school. The director of the Mary Baldwin College project 
(p. 9), however, concluded after his own research that academic rather than motivational 
deficiencies caused GED holders to drop out of college. The rigorous program that he 
designed for inmates of a women’s prison resulted in unprecedented levels of achievement and 
retention. 

This project and the University of Virginia’s Core Knowledge Foundation (p. 1 19) reconfirm 
the findings of many FIPSE projects— that ill-prepared or poorly-performing students learn 
best if they are provided solid academic content in an atmosphere of high expectations. For 
some students, however, the academic program needs to be buttressed with personal and 
financial assistance, as shown by Western Washington University’s Law and Diversity 
Program (p. 17) and Prescott College’s Center for Indian Bilingual Teacher Education 
(p. 195). This assistance works especially well when it takes into account the needs of the 
particular group. California State University, Fresno’s project on academic literacy skills 
(p. 31), for example, includes information on academic culture in addition to language, 
because many of its students are the first in their families to attend college. 

This volume abounds in lessons for international projects. In efforts that involve study abroad 
and collaboration with other institutions, simple is better. As the University of Connecticut’s 
EuroTech (p. 155), San Diego State University’s MEXUS Program (p. 175), and the 
University of Michigan’s Business Assistance Corps (p. 169) demonstrate, the administrative 
complexities of such projects can become overwhelming. Getting students to go abroad and 
stay there long enough to profit from the experience poses significant problems. Projects such 
as EuroTech, where the stay abroad is built into the curriculum and is not likely to delay 
graduation, have the best chance of success. 




The primacy of the faculty’s role in educational reform is obvious to anyone reading these 
pages, as is the need to recognize and honor this role. Projects large and small— from the 
College of William and Mary’s efforts to reform its advising system (p. 147) to the Missouri 
Coordinating Board for Higher Education’s establishment of performance funding across state 
institutions (p. 21 1)— succeeded in great part because of the amount and quality of attention 
they paid to faculty. 

Even the best-designed projects, however, occasionally run into faculty resistance. The 
American Association for Higher Education’s Forum on Faculty Roles and Rewards reported 
greater openness to reforms on the part of administrators than on the part of the very faculty 
who were supposed to benefit from them. No matter how outstanding the innovation or how 
carefully it is introduced, resistance often crops up. California State University, Fresno, the 
University of Medicine and Dentistry of New Jersey, the Dickinson College Cluster, 

Cleveland State University — all these highly successful projects encountered some measure of 
resistance. The lesson for those contemplating a substantial innovation is to expect faculty 
skepticism, accept it as evidence of intellectual liveliness, and get on with the reform. 

This volume of Lessons Learned, as did the three that preceded it, contains its share of 
surprises. Sometimes students reacted unexpectedly to well-intentioned efforts to help them— 
some refused to be mentored; others felt that faculty should be less lenient. An impeccably 
designed evaluation of a highly regarded faculty development method could not show that the 
method resulted in improved performance on the part of the students of participating faculty 
(Seton Hall University, p. 53). Evaluation of technology proved especially problematic. The 
staff of Tufts University’s software evaluation project (p. 65) found it difficult to pinpoint how 
technology affects student learning. Projects using technology and discovery learning 
highlighted the differences between students who want to explore a subject for its own sake 
and those who want to get through the material as quickly as possible (see Tufts University’s 
Perseus project, p. 1 13, and the Dickinson College Cluster, p. 247). 

At California State University, Fresno (p. 31), on tests of academic language proficiency 
native English speakers performed no better than students with limited knowledge of English. 
And staff of California State University, Los Angeles’s Project LEAP (p. 141) found that 
language-enhanced courses intended for non-English speakers benefited their English-speaking 
peers as well. Some reforms, such as Clemson University’s Cooperative Chemistry 
Laboratories and Diablo Valley College’s interdisciplinary physics and calculus curriculum, 
which were designed for a general student population, unexpectedly resulted in better learning 
and retention on the part of minorities and women. 



All this strongly points to the conclusion that good teaching helps all students. But it is 
crucial to keep in mind that even projects with unassailable evidence of success can founder on 
the shoals of fiscal realities. Initiatives involving interdisciplinary teaching, extensive faculty-, 
student contact, innovative uses of technology and other hallmarks of teaching excellence are ■ 
often expensive. Although a tiny minority, some of this volume’s most illustrious entries did 
not continue because of lack of funds. 

Fortunately, these are the exception. Following in FIPSE’s almost 30-year tradition, the great 
majority of the projects featured here were remarkably successful in generating additional 
funding from both institutional and external sources after the nPSE grant expired. 

Despite a few anomalies, the lessons gleaned from the projects described below are 
encouragingly familiar; that holding students to high standards works, that passionate, creative 
teaching is crucial, that difficulties will arise and can be surmounted, that excellent work 
eventually brings recognition and reward. As good educators know, the wheel of innovation 
eventually returns to confirm the wisdom of the ages. FIPSE is proud of its role in that cycle. 

Eulalia B. Cobb 
Program Officer 
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MACALESTER COLLEGE AND MOUNDS VIEW PUBLIC SCHOOLS 
A Computer-based, Articulated Geography Curriculum K-14 



Purpose 

GeoLinks evolved out of a need for an outcomes-based, K-14 curriculum in geography 
that could be offered via electronic technology. Its creation coincided with three major 
developments in geographic education: the National Geography Standards, the Minnesota 
High School Graduation Standards, and new entrance requirements for the Minnesota State 
System of Colleges and Universities. 

Macalester College, in partnership with Mounds View Public Schools, produced an 
electronic geography curriculum that not only conforms to national and state geography 
standards but was embraced by classroom teachers, allowing them greater flexibility in 
designing classwork that matches the standards. 

Innovative Features 



Because the high school outcomes had been previously established, the task was to 
determine GeoLinks’ scope and sequence, i.e., the appropriate grades at which to introduce 
geographic concepts, and how best to reinforce them so that students would master the 
outcomes by the time they graduated. 

GeoLinks consists of a set of 128 testable outcomes tied to the National Geography 
Standards— arranged in a K- 12 sequence and defining what a student should know and be able 
to do after completing a high school geography course— and a diverse collection of geography 
lessons, units, and resources. 



Most of the GeoLinks lessons are interdisciplinary, providing lessons about geography 
together with mathematics, science and reading. They were written and pretested in 
collaboration with hundreds of teachers from Minnesota school districts in the suburbs, the 
inner city, and rural regions, and with Russian teachers from Moscow State Pedagogical 
University. Writing the lessons required a prior evaluation of the preparation in geography 
that Minnesota teachers typically receive. 

The creators of GeoLinks strove to include lessons that would also be useful to social 
studies teachers who have never taught geography before and to students with special needs. 
For teachers who do not have computers equipped with compact disc (CD) players, a special 
feature allows them to download portions of the curriculum from the CD to a diskette. 

The GeoLink design allows teachers to locate lesson plans for teaching a particular set 
of outcomes to diverse groups of students. Its HyperCard software lets teachers use a few 
simple computer commands to sort through 1 ,022 geography lessons in 30 seconds or less. 

The lessons cover all grades, from kindergarten through high school, and include worksheets, 
quizzes, special readings, maps and color slides. They can be used by school districts creating 
new curricula, by teams of teachers developing shared lessons, or by individual teachers 
shaping their own year-long curriculum. 

The lessons follow a standardized format, including objectives, classroom materials, 
teacher preparation, activities guides, and outcomes evaluation. Not only may teachers select 
lessons according to the national and state standards, but also according to learning style, cognitive 
level, grade level, key word, continent, intended outcomes, or a combination of these. 

If, for example, a teacher would like illustrative lessons on post-Soviet Russia, she can find 
appropriate maps and slides with just a few key strokes. Once a teacher selects a standard, all the 
learner outcomes related to that standard are displayed on the screen. She selects the outcome that 
best fits her interests and local setting, starts a search, and waits while GeoLinks locates lessons for 
that outcome. Then she can either print out the lessons or copy them onto a diskette. This feature 
of “just-in-time publishing” allows teachers to print out only those worksheets or illustrations they 
need for a certain class from among the vast amount of information stored on the GeoLinks CD. 
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Evaluation and Project Impact 



In 1994-1995, 1,978 students in twelve middle schools participated in an evaluation of 
GeoLinks as a classroom tool, using four schools as controls. GeoLinks users and non-users 
were given pre- and posttests in their geography courses based on the standardized test for 
intermediate grades developed by the National Council for Geographic Education. An 
item-analysis was conducted of the number of incorrect answers for each school, the difference 
in the class averages before and after the course, and the percentage difference of correct 
answers. 

The analysis of the pre- and posttests turned up dramatic differences in student 
performance between those classes that used GeoLinks and those that did not. Correctly 
answered questions rose seven percent in GeoLinks classes and only two percent in 
non-GeoLinks classes. 

Lessons Learned 



Even though project staff put more effort into the project than they had originally 
anticipated, several problems hampered progress. The writing, editing, entering and proofing 
of the lesson plans proved to be an arduous task because the lessons were written by a wide 
range of teachers and the quality of the lessons reflected that range. Some of the teachers 
originally selected to write lesson plans either failed to produce them or submitted 
unacceptable ones— only half the lessons written for GeoLinks were suitable for inclusion. 

The apparent lesson about lesson plans is that it is necessary to recruit teachers with substantial 
experience and writing abilities. 
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Project Continuation and Recognition 



In 1995, the Minnesota Association of Supervision and Curriculum Design named 
GeoLinks an “Outstanding Curriculum Project,” and it was selected for inclusion in a special 
exhibition of innovative curricula at their national convention, 

GeoLinks is now self-supporting, having been nationally disseminated without 
additional grant funds from FIPSE or other federal agencies. The fees from GeoLinks site 
licenses enabled the project to maintain a half-time staff associate to continue the diffusion 
effort during 1997, 

The curriculum was originally designed to be operated by the HyperCard software 
package that is available at no cost for all teachers who have Macintosh computers. To 
broaden the curriculum's usefulness, GeoLinks has been redesigned to operate on a Web site 
and thereby be accessible through either Macintosh or Windows machines. New lessons have 
been added to this edition, and there are plans to produce low-cost simplified, diskette versions 
of GeoLinks by grade level for individual teachers. 

In 1998, a summer institute for teachers interested in teaching advanced placement 
geography continued to articulate the electronic geography curriculum from elementary grades 
to college by producing a model AP course for students. 

The gap between K-12 and postsecondary geography instruction was identified in the 
initial grant proposal. Faculty from Macalester College and Minnesota State University, 
Mankato, ran faculty development seminars for geography professors and social studies 
teachers to build a connection between the K-12 curriculum and first year college courses. 

This resulted in a new FIPSE- and National Science Foundation -funded program for preservice 
teacher training at Minnesota State, 
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Dissemination 



At this time, approximately 800 sites across the nation have purchased licenses for the 
curriculum, which is available to every teacher at these sites. Several states have purchased 
licenses to use the curriculum in their resource collections. The number of teachers using the 
CD could not be monitored easily. However, assuming that at least two teachers taught 100 
students per year for three years in these 800 schools, at least 480,000 students have used 
GeoLinks lessons. 

Information about GeoLinks has been communicated to diverse educational audiences 
through numerous national and state conventions, such as those of the National Diffusion 
Network, the National Geographic Society, the National Council for Geographic Education, 
the Minnesota Education Association, the Minnesota Council for Social Studies, the Minnesota 
Educational Computing Corporation, and many others. 

GeoLinks was also disseminated through direct mail, telephone, e-mail, press releases, 
training sessions, interviews, and articles, and was featured in every recent issue of The 
Geography Connection, the newsletter of the Minnesota Alliance for Geographic Education. 
The National Geographic alliances have included GeoLinks in their resource collections and 
summer institutes, and it now has its own Web home page. Free workshop demonstrations 
have been offered to Twin Cities schools. 

Teacher training with GeoLinks has been incorporated into GeoTeach, the 
FIPSE-funded preservice training project underway at Minnesota State University, Mankato. 
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Available Information 



Articles about the new curriculum have appeared in the Journal of Geography, the 
MEA Advocate, Geography in Schools, and National Geographic Update. Information about 
other GeoLinks-related articles, catalogs, and multimedia and videodisc compendiunis for 
education and training is available from: 



David A. Lanegran 
Department of Geography 
Macalester College 
St. Paul, MN 55104 
Telephone: 651-696-6504 
e-mail: Lanegran@macalester.edu 

or 

Patrice A. St. Peter 

Irondale High School 

2425 Long Lake Road 

Mounds View Public Schools 

New Brighton, MN 55112 

Telephone: 651-586-5200 

e-mail : PATSI@mail . mounds view . k 1 2 . mn . us 

The address for the GeoLinks Web site is: mage.geog.macalester.edu/GeoLinks/ 
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MARY BALDWIN COLLEGE 

Development of a Series of Liberal Arts Courses to Strengthen the College Performance 

of High School GED Graduates 



Purpose 

In 1988 Mary Baldwin College, in collaboration with the Virginia Department of 
Correctional Education, began to offer college courses leading to a B.A. at a women’s prison. 

Soon, however, high rates of early attrition became apparent, primarily among holders of the 
General Educational Development (GED) certificate. In the first four years of the B.A. 
program, 45 percent of the GED students withdrew during the first semester for reasons other 
than parole. After three semesters, only 38 percent of GED students persisted. 

This was not unusual. Although research on GED tests shows that they function well as 
evidence of readiness for work, most students now take them for college admissions, and college 
attrition among GED holders ranges between 60 and 90 percent. The grade point average of GED 
students who persist to graduation, on the other hand, does not differ significantly from that of 
other students— probably because the massive early attrition weeds out all but the most able. 

The high college attrition among GED students was attributed in the literature to the 
same low motivation which had caused these students to withdraw from high school. But the 
project director suspected that this was not the case for the Mary Baldwin students, because 
motivation is generally high among prison inmates. 

It seemed plausible that the academic preparation of GED students was related to their lack 
of success in higher education. The passing standards for the GED, which vary from state to state, 
are often quite low. In Virginia, it is possible to pass the mathematics test while failing all algebra 
and geometry questions, and to receive no credit on the essay and still pass the writing test. 
Knowing this, some preparatory programs omit instruction in these topics altogether. 

The project director felt that it was unreasonable to expect two or three months in even 
the best GED preparation program to replace two or three years of daily academic work. A 
look at the characteristics of students who persisted versus those who dropped out of the prison 
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program showed that, indeed, the determining factor was dropping out of high school prior to 
entering the 12* grade. Clearly it was academic rather than motivational factors that needed to 
be addressed to help GED students to succeed. 

Innovative Features 

The project— funded by FEPSE along with the American Bar Association, the Presbyterian 
Women’s Thank Offering, and the Titmus, Kates, and Beasley Foundations— consisted of three 
courses taken by GED students concurrently with initial college enrollment. 

In creating the courses, project staff made a number of assumptions regarding the academic 
shortcomings of GED preparation. They expected students to have no experience in critical reading 
of and writing about book-length texts. They assumed little knowledge of world literature and 
history, algebra, geometry, and probability, little preparation for testing other than multiple choice, 
and no experience with classroom discussion or interdisciplinary thinking. 

The three resulting courses concentrated on specific academic skills and content and 
demanded more reading and writing than many college courses. They were as follows: 

1 . The Ancient and Medieval Worlds. This course examined the roles of men and 
women at different historical times and in different cultures and different views of the “virtue” 
of patience. Students read The Odyssey, Ruth, Job, Plato’s dialogues about the death of 
Socrates, Murasaki Shikibu’s Diary, Kamo Chomei’s Hojoki, The Canterbury Tales, and a 
play by Shakespeare. Writing, which was taught by the traditional rhetorical approach, 
included eight short essays, several tests, and two oral reports. 

2. Modem American Voices. This course focused on race and social class in American 
culture in the context of the relationship between the social sciences and the humanities. Students 
read The Adventures of Huckleberry Finn against the background of the political rhetoric of Lincoln 
and Douglas. They read Flannery O’Connor, Toni Morrison, Alice Walker, and Lee Smith in the 
light of sociological theory on race, gender and family. They read the lyric poetry of Emily 
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Dickinson, T.S. Eliot and Gwendolyn Brooks, and the speeches of the Civil Rights movement. 
They wrote essays on these texts and developed one into a research paper. 

3. The Scientific Method. This course, which began with Descartes’s D/jcowrje on 
Method, investigated the mathematical and philosophical underpinnings of the natural sciences. 
It examined basic algebraic functions and formulas as they relate to chemistry; the course also 
examined probability theory as it applies to genetics and intelligence and graphs and statistics. 
Students read The Double Helix and The Mismeasure of Man. 

Students could begin the course sequence at any point. Because of high retention, there 
were soon more applicants than the courses could accommodate, so they were admitted on the 
basis of conduct, which allowed project staff to observe students with a wide range of 
preparation and GED test scores. 

The course sequence was also offered, on a one-time non-credit basis, to GED students 
enrolled at three local community colleges. 

Evaluation and Project Impact 

The retention of 89 GED holders during the three years following the inauguration of 
the project course sequence at the prison was compared to that of GED students enrolling in 
the college program’s first four years. Prior to the project courses, first-semester GED 
students consistently withdrew at rates between 42 and 52 percent. During the project period, 
on the other hand, first-semester GED students withdrew at rates between zero and nine 
percent. 

Eighty-five percent of GED enrollees in the project completed the fourth semester, 
compared to only 34 percent during the first four years of the collegiate program. In addition, 
the first-semester GPAs of project students were much higher than in previous years (2.34 
versus 1.59). (The improved GPA was not solely dependent on the project course grade— 
when that grade was removed, the GPA still averaged 2.27.) 
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At the community colleges, a comparison group of 48 GED holders volunteered to 
make their enrollment data available. Eighty-five percent of the 39 project students completed 
the first semester, compared to only 50 percent of the control group. By the end of the third 
semester, 74 percent of the project students were still enrolled, versus 42 percent of the control 
group. Although the effects of the project courses were less remarkable with the community 
college population, they nevertheless conferred a retention advantage of approximately 35 
percent on their students compared to the controls. 

Among the prison population, by the third semester there were few differences in 
achievement and retention between project students and their predecessors, bearing out the 
research that shows that GED students who persist do relatively well in later semesters. 
However, a comparison of the total number of women enrolled in college — including both 
those who were still in and those who had left prison— showed that 90 percent of the women in 
the project courses were continuing in higher education three semesters after initial enrollment, 
compared to only 38 percent of those who had not participated in the project. 

The project course helped significantly with mathematical and writing skills as well. 
Students took the quantitative section of the Scholastic Assessment Test (SAT) at the beginning 
and at the end of the Scientific Method course. Their scores increased from a mean of 331 — 
well below the level required for college courses— to 457 (the median, 450, is the minimum 
required by Mary Baldwin for admission to college algebra). 

A random sample of student essays from the project courses was rated by a group of 
experienced English teachers on a scale of zero to ten, on criteria ranging from syntax to 
spelling and content. First semester students scored an average of 3.4; second semester 
students, 5.9; and third semester students, 7.7. 

Participation in the program also correlated with lower recidivism. In Virginia, 49 percent 
of released women prisoners return to prison within two years (for those without a high school 
diploma, the rate is 60 percent). Twenty-nine percent of women who enrolled in a project course 
but dropped out before completing the first semester were reincarcerated, while under 1 1 percent of 
those who completed one or more semesters returned to prison within two years. 



Although during their first semester students received as many disciplinary sanctions as 
they had during the previous three months, in later semesters their misconduct levels decreased 
by more than half. This came as a surprise to prison staff, who had expressed concern that 
college students would become disciplinary problems. 

Persistence in the project courses seemed to decrease the dropout rate of the women’s 
children as well. The children of GED-holding incarcerated mothers dropped out of school at 
a rate of over 32 percent, and the children of mothers who did not complete their first semester 
in the project dropped out at a rate of 35 percent, but the children of those who completed at 
least one semester in the project left school at a rate of only 14 percent. 

Lessons Learned 



Research shows that many GED students are quick to become discouraged and abandon 
their studies for good. A successful first semester, on the other hand, often leads to continued 
enrollment, and the project’s retention figures reflect this. 

The higher retention among the imprisoned students may be due in part to less reliance 
on adjunct faculty and to the use of conduct as an admissions criterion. Most significantly, 
however, in prison there were few competing demands and few alternatives, whereas in the 
community colleges the hurry-up-and-get-through-it mentality that often prevails among adult 
students caused many to choose less demanding classes. 

Student evaluations of the Ancient and Medieval Worlds and Modem American Voices 
courses were highly positive, although less so for the sections on T.S. Eliot and political 
rhetoric. The Scientific Method course had to be substantially revised to provide more 
non-mathematical context and thus motivate students to learn the mathematical content. In 
retrospect it appears as though, given the quantitative deficits of these students, the sequence 
should contain two mathematics courses. 
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Students showed a startling lack of preparation in algebra— not one of those in the first 
offering of the Scientific Method Course could solve either x+2=4 or 2a=4 in a pretest. 

They experienced high math anxiety and thought that mathematics was something done for its 
own sake, with few applications to the real world. 

The magnitude of the academic deficits of GED students was such that at first it 
prevented them from putting their life experience to use in the classroom. They were 
incapable of participating in class discussion. They failed to relate opinion to fact and took 
disagreements personally, often storming out of class in anger. Reading primary texts and 
writing about them proved overwhelming and so did examinations— students could not answer 
identification questions with complete sentences or organize responses to essay questions into 
paragraphs or even sentences. They could not relate texts read in class to outside sources or 
ideas, and when they eventually began to do this, the papers became, for a time, chaotic. 

Higher education seems all too willing to respond to the demands of adults who want 
college credentials quickly and with as little effort as possible. Students with academic ability 
and considerable life experience may succeed in such programs, but clearly few GED students 
can. They require a slower track with, at first, lowered expectations to ensure success. 

Project Continuation 

The college’s B.A. program at the prison and the community college program have 
been canceled due to lack of funds. The three project courses, however, continue to be 
offered at the prison under a contract with the Virginia Department of Correctional Education. 
Supplemented with workshops on college admissions, financial aid, and college-related skills, 
the courses are available to women who will be released within a year. Evaluation is 
continuing. 
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Dissemination and Recognition 



The program has been the subject of a number of presentations in the United States and 
Canada. 



Available Information 



Additional information may be obtained from: 

Ashton D. Trice 
Department of Psychology 
MSC 7401 

James Madison University 
Harrisonburg, VA 22807 
Telephone: 540-568-8189 
e-mail: tricead@Jmu.edu 
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WESTERN WASHINGTON UNIVERSITY 
Law and Diversity Program 



Purpose 

It is a documented fact that in the United States people of color and other minorities are 
both underrepresented in the legal system and have little faith in it. To achieve proportional 
representation in the legal field today, the number of Asian American lawyers would have to 
double, the number of Hispanic lawyers more than triple, and that of African American and 
Native American lawyers quadruple. As minority populations all but double over the next 50 
years, their underrepresentation in the legal profession may worsen dramatically. 

The Law and Diversity Program seeks to meet this need by recruiting minority and 
non-traditional students who have either not considered entering the legal profession or who 
need to improve their skills in order to gain admission to law school. The program provides 
an academically demanding environment where the students’ perspectives are valued and their 
needs for various kinds of support are met. 

Innovative Features 



The Law and Diversity Program was created at Fairhaven College, an interdisciplinary 
college of Western Washington University dedicated to innovative approaches to education. 
Most Fairhaven students design their own academic concentrations, a structure that easily 
accommodates the project’s pre-law curriculum. 

The program recruits non-traditional and minority students—those whose perspectives and 
experiences have been traditionally underrepresented in law schools, the legal profession and the 
law. Its graduates include African Americans from crime-filled urban neighborhoods, a 
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Chamorro woman from the Northern Mariana Islands who became the first native lawyer on 
her island, a refugee from the civil war in Eritrea, former gang members, single mothers, 
mature students, students whose backgrounds include poverty, rape and abuse, and activists 
from the disabled and gay communities. 

When they begin the program after completing their general education requirements, 
students become members of a two-year learning community. Law and Diversity Program 
students often show academic promise but have marginal records and tend to give up when 
studies become demanding. The program’s cohort system helps them to succeed despite the 
rigorous material, while the common curriculum allows them to form close ties with faculty 
and to synthesize their learning. 

Given their adverse personal and financial circumstances, many of the program’s 
students require support, both emotional and economic, from faculty, from mentors and role 
models, and from each other. Faculty coordinate efforts with the financial aid office, the 
registrar, the counseling center and the writing center to ensure that help is available. The 
admissions office assists in recruitment, and the development office raises funds (for 
emergency loans, scholarships and other forms of assistance) from private individuals and local 
law firms. 

The curriculum consists in part of core courses, taught by two instructors who share a 
full-time appointment. These instructors also teach the integrative seminar, serve as “master 
learners,” and offer advice and personal support to students. The rest of the curriculum is 
made up of existing courses in a number of different departments, which Law and Diversity 
students take as a group along with the general university population. 

Before designing the program, faculty consulted with representatives of local law 
schools and were advised that it was more important for the curriculum to emphasize verbal, 
analytical and research skills than to teach a specific body of knowledge. Accordingly, faculty 
selected challenging, skills-intensive courses that introduce students to the U.S. legal system, 
familiarize them with basic legal concepts and terminology, and place the law in its social, 
historical and political context. 
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The curriculum includes, among others, courses on American legal and political 
systems, political economics, race, politics and public policy, and logic and problem 
solving. The constitutional law course has proven particularly effective for law school 
preparation. In it students use Gerald Gunther’s constitutional law casebook in small 
question-and~answer groups. 

The quarterly integrative seminar is designed to synthesize knowledge, discuss law school 
admissions and focus on specific skills. The last seminar deals with the law as a profession. 

The curriculum not only demands considerable amounts of writing and critical thinking 
but also allows students to participate in one or two mock arguments and have exposure to 
basic legal research. The courses vary from year to year but their challenging nature and their 
focus on communication skills remain constant. 

TTie final quarter consists of an inteaisive internship. Studaits woik as clerks in law firms, courts, and 
city, state and federal legal offices. They also serve in investigative positions with the public defender’s 
office, as legal advocates for victims of domestic violence, and in other positions at the American Civil 
Liberties Union, lawyer referral services, disability rights groups and juvenile detention facilities. 

Students receive various fonns of help in applying to law school. Faculty make pertinent 
materials available and arrange for students to visit nearby law schools and attend classes. Because 
progrmn faculty know their students well, they are able to write detailed letters of recommendation. 
Because many students apply with below average numerical indicators, the letters address the 
students’ actual performance on relevant topics. A commercial provider of an LSAT preparation 
course makes it available to each cohort for a fraction of the $795 fee. 

Evaluation and Project Impact 

The efforts to gauge the effects of the project included student self-assessments, focus 
group assessments by faculty and students, writing portfolios, LSAT preparation and scores, a 
video documentary, retention and academic performance data, and interviews of all 29 alumni 
and of faculty and staff of the law and graduate schools in which they enrolled. 
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Project staff compared the students’ diagnostic LSAT scores, taken prior to the start of 
the preparation course, to the final scores. The mean score for students in the two cohorts 
went from 136 to 145, and from 139 to 146 respectively, raising the mean percentile for both 
classes 19 points. For the students who went to law school, the improvements were even more 
dramatic, with percentiles rising by as much as 25 points. 

Retention statistics are also encouraging. Of the 29 students participating in the Law 
and Diversity Program, 93 percent completed the program and 96 percent the bachelor’s 
degree. This compares quite favorably with the university’s minority junior-to-senior retention 
of 54 percent for those who entered as freshmen, and 48 percent for transfers. 

Of the 29 students tracked, 16 applied to law school and 12 were admitted, and of these, by 
the end of the project, all but one were still attending or had graduated. Two students went to 
graduate school, and five obtained law-related jobs. The rest were studying to retake the LSAT and 
pursuing other interests and jobs. Most intended to continue their education. 

Students felt “satisfied” to “highly satisfied” with the program and thought the cohort 
experience valuable, despite conflicts that arose in the second cohort. They appreciated the 
faculty’s care and support but criticized what they felt was their excessive leniency. Although 
they thought that it needed more focus and stnicture, students found the integrative seminar 
useful. 

Faculty and staff at the graduate and law schools that graduates of the program attended 
stated that the students had been well prepared. Indeed, most were performing above the 
traditional numerical predictors of academic success. 

Law students felt that the program had trained them adequately, although they stated 
that more work on specific skills— such as methods of case analysis— and more counseling on 
the isolating and competitive nature of law school would have been helpful. One student, who 
had the lowest LSAT score in her entering class, submitted the top paper on the first legal 
writing assignment and, at the end of the first year, ranked 90th out of a class of 216. 

Another student, with LSAT scores in the 55th percentile, finished the first year in the top 
15-20 percent of the class. 
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Many alumni are active in community service, working with youth athletic programs, 
juvenile offenders, victims of domestic violence, migrant farm workers, homeless youth, 

AIDS education in minority communities, and gay and disability rights issues. Students 
attending law school become involved with campus organizations — some of which they have 
started themselves. They also study abroad, write for law reviews and work as interns and 
clerks. 

In 1996 the Council of the American Bar Association’s Section of Legal Education and 
Admissions to the Bar issued a statement regarding preparation for legal education that echoes 
many of the provisions of the Law and Diversity Program. The assertion that “difficult 
courses from demanding instructors is the best generic preparation for legal education” 
especially reflects the convictions of the creators of this project. 

Lessons Learned 



Project staff did not find it easy to recruit students of color into the program, because 
the community where the university is located is predominantly white, and the student 
population is only 13 percent minority. This also made it difficult to find local minority 
lawyers and judges and firms to provide internships. Fortunately, lawyers and judges in 
metropolitan areas of the state readily offered assistance. 

From the beginning, discussion of the issues surrounding diversity was a fundamental 
feature of the program. It soon became obvious, however, that students did not spontaneously 
understand the commonalities among all forms of oppression and tended instead to engage in a 
comparison of oppressions, with the cohort eventually fragmenting into groups segregated by 
race, gender and sexual preference. In the future, much of the first quarter will be devoted to 
helping students develop the ability to understand and support each other despite their 
differences— a skill which is being increasingly recognized by law schools as essential for good 
lawyering. 
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Instead of receiving standard grades, in some Law and Diversity Program classes 
students wrote narrative self-evaluations to which the instructor responded. It soon became 
apparent that students were putting more effort into their regularly-graded classes because they 
assumed that law school admissions relied heavily on grade point averages. In fact, law 
schools did not view self-evaluations as problematic. Staff informed students of this and 
instructed them on how best to present their self evaluations in applications. They are also 
providing an application cover sheet explaining the purpose and use of self-evaluations. 

Despite the virtues of the cohort system, staff became aware of the need to better 
integrate Law and Diversity Program students with the rest of the campus, and to offer special 
advising to those who, while enrolled in the program, discover that law school is not for them. 
In response to students’ desire to pursue areas of individual interest, staff decided to grant 
later cohorts flexibility in the second year to take several courses outside the Law and 
Diversity curriculum. 

To better accommodate law school application cycles, faculty changed from the 
traditional fall-winter-spring academic year to a winter-spring-fall Schedule. Applications are 
normally prepared in December, which, under the traditional system, gave faculty just four 
quarters to work with students prior to writing letters of recommendation: In the modified 
year, on the other hand, students have nearly finished the sixth and final quarter by the time 
they apply to law school. (After completing the Law and Diversity Program, students may 
take further courses if they choose.) 

Project Continuation 

The project has continued since the expiration of FIPSE funding, with the third cohort 
continuing the high retention of its predecessors and graduating in June of 1997. The 
university converted the program director’s full-time faculty position to tenure-track. Two 
other faculty members also participate in the program. 
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Dissemination and Recognition 



Faculty and students made presentations at a number of conferences, and their remarks 
led to discussions about adapting the Law and Diversity Program to other campuses. 
Audiences generally felt that the model’s interdisciplinary requirements would preclude its 
being adapted in its entirety but perceived the cohort and skills development approaches as 
having considerable potential for replication. While the program at Western Washingtoh 
University focuses on pre-law preparation for minority and other disadvantaged students, staff 
believe that the model, or parts of it, can work for any group of students. 

The project received Western Washington University’s Diversity Achievement Award. 

Available Information 

Further information may be obtained from: 

Marie Eaton 
Fairhaven College 
Western Washington University 
Bellingham, WA 98225-9118 
Telephone: 360-650-3680 
e-mail: eaton@henson.cc.wwu.edu 
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WESTERN INTERSTATE COMMISSION FOR HIGHER EDUCATION (WICHE) 
Balancing Quality and Access: Reducing State Policy Barriers to Electronically Delivered 

Higher Education Programs 



Purpose 

Problems of access to higher education are most often associated with issues of race or 
economic status. In the western states, however, these problems are compounded by 
geography. There, even well prepared students are often stymied by the difficulty of getting 
themselves to the classroom where what they want to learn is being taught. In the case of 
individuals from disadvantaged backgrounds, the added obstacles of distance and weather often 
prove insurmountable. 

It is hardly surprising, then, that western states have emerged as leaders in distance 
education, developing a sophisticated telecommunications infrastructure over the last two 
decades. The approach to distance learning, however, varies greatly from state to state. Some 
states envision distance learning strictly as a means of serving students in outlying areas, while 
others have created major statewide programs and still others have specifically designated 
certain institutions as providers of distance education. 

The policies regulating this multitude of options, especially when offered by 
out-of-state providers, are just as varied. While some states require only registration of 
out-of-state programs, for example, others insist that such programs be licensed. Many others 
waive the licensing requirements if the programs are offered by an accredited institution. To 
complicate matters further, when the Balancing Quality and Access Project was conceived 
there existed no agreed-upon criteria for assessing distance learning, and the procedures of 
regional accreditation associations and state regulatory bodies were not coordinated. 

Clearly, if students were to take full advantage of the flood of advances in distance 
education, it was necessary to address the barriers imposed by interstate policy, and especially 
the need for standards with which to judge the quality of these programs. That is what this 
project intended to achieve. 



Once the project began, however, it became apparent that the question of interstate 
policies was less crucial than it had first appeared. For while it is theoretically possible to 
control the offerings of an out-of-state institution if they are received at, for example, a local 
community college, at home students can have unlimited access to programs on cable 
television or on the Internet. As a result, many states accept all out-of-state programs offered 
by accredited institutions. 

In addition, the decision by the Western Governors’ Association to develop a virtual 
institution, the Western Governors’ University, offered a means of addressing the issue of 
interstate compatibility in the future. The university, which will eventually award its own 
credentials based on outcome assessments, will function as a regional broker of courses, 
learning rnodules, and degree programs offered by colleges, universities and companies, thus 
eliminating all barriers to interstate distance education. 

At the same time, as they began to play a more important role in helping states to 
approve oiit-of- state distance education (12 of the 15 WICHE states used accreditation as the 
principal criterion for granting approval), the regional accrediting associations became more 
interested in assessing the quality of distance education. The accrediting agencies looked to 
this project to help establish specific distance education standards. 

While the concern over interstate regulatory policies diminished, however, the need to 
ensure quality in programs made available through technology became preeminent. The 
project’s creators realized that their most significant task would be to help learners to evaluate 
and select distance education programs. Hence, the project’s new focus became such questions 
as whether or how a student engaged in distance learning would have direct access to faculty, 
what kind of academic advising and support services would be available to him, and whether 
he could count on the originating institution’s commitment to offer the program for as long as 
he required it. 
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Innovative Features 



The project’s originators began by documenting existing state regulations and practices 
about degree programs delivered electronically from out of state. They surveyed and 
interviewed personnel of state higher education regulatory and governing offices and reported 
the results in a publication entitled When Distance Education Crosses State Lines: Western 
States' Policies. 

Project staff then proceeded to develop the “Principles of Good Practice for 
Electronically Offered Academic Degree and Certificate Programs.” They achieved this 
through a consultative process, circulating and revising drafts in the light of comments from 
the WICHE Advisory Board, the regional accrediting agencies, and state higher education 
executive officers in the western states. State higher education agencies in turn distributed 
drafts to local colleges and universities, and WICHE disseminated preliminary versions of the 
principles through its paper and electronic media. 

The principles, which address entire programs rather than individual courses, are based 
on the assumption that the institution and its electronic curricula are accredited by a nationally 
recognized accrediting body. The principles cover curriculum and instruction, evaluation, and 
institutional context and commitment— including institutional role and mission, faculty support, 
resources for learning, student services, and commitment to distance learning programs. 

Project Impact 

The Principles of Good Practice have become the basis for national agreement on the 
standards for education delivered through technology. They have been endorsed by state 
agencies, regional higher education groups, and the regional accrediting associations. 

Eight western states— Arizona, Colorado, Idaho, Montana, New Mexico, Oregon, South 
Dakota and Washington— have in various ways incorporated the principles into their review and 
approval processes for in-state programs. The Western Legislators’ Conference passed a 
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resolution encouraging member states to consider adopting the principles as the basis for 
in-state assessments. The Western Governors’ University will stipulate that all whose courses 
and programs are to be listed in the WGU online catalog must certify their adherence to the 
principles. 

The impact of the principles has been felt outside the West as well. The Southern 
Regional Education Board adopted a set of principles for participation in its Electronic 
Common Market with wording almost identical to the Principles of Good Practice developed 
by this project. The regional accrediting associations’ national Task Force on Distance 
Learning endorsed the principles, as did all individual regional accrediting commissions, most 
of which included the principles in their handbooks. 

Seventeen out of 19 professional associations contacted by project staff requested 
information and are considering incorporating the principles into their standards policies. The 
National Association of State Approving Agencies also adopted the principles’ basic 
philosophy, and Virtual College, a publication designed to accompany Peterson’s Guide to 
Colleges and Universities, endorses the principles as tools forjudging quality. 

Lessons Learned 



California, whose rules governing out-of-state providers have long been among the 
strictest in the West, is closing down its state licensing and regulatory agency. This 
development confirms the project staff’s belief that across the nation future efforts will focus 
on educating students about their choices rather than on attempting to regulate program 
providers. 
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Dissemination and Recognition 



It was a major goal of the project to have the principles become the standard for 
discussions of quality in distance education, both in the West and throughout the country. To 
ensure this, staff mailed a one-piage brochure on the principles to WICHE commissioners, to 
other regional higher education organizations, to state higher education executive and 
administrative officers, to college and university presidents, and to the initiators of the Western 
Governors' University. 

Project staff published articles in Change and other magazines, made conference 
presentations, contacted professional accrediting associations, and educated students with two 
booklets, entitled Distance Education: A Consumer's Guide — What Learners Need To Know, 
and Good Practices in Distance Education. Numerous requests for copies of the principles led 
to a second stage of dissemination as recipients incorporated the principles into their own work 
and republished them in journals and newsletters. 

Available Information 



Additional information may be obtained from: 

Sally M. Johnstone 

Western Cooperative for Educational Telecommunications 
P.O. Box 9752 
Boulder, CO 80301-9752 
Telephone: 303-54 1 -0232 



CALIFORNIA STATE UNIVERSITY, FRESNO 
Assessment of Academic Literacy Skills: Preparing Minority and Limited English 
Proficient (LEP) Students for ‘Postsecondary Education 



Purpose 

Despite important recent strides in diversifying the means of instruction, college students 
still acquire most of their education by listening to lectures and reading textbooks. Yet the language 
of lectures and textbooks is radically different from that spoken on a daily basis by most students 
and their families. This difference is often at the root of serious academic difficulties. Academic 
language, which includes words and phrases common to all fields, is used in lectures and texts to 
teach the technical information of each discipline. Because the elimination of books and lectures is 
neither feasible nor desirable, we can expect that students who come to campus already familiar 
with academic language will continue to enjoy a clear advantage over those who do not. 

Students with academic language deficiencies come from varied backgrounds. Some are 
native English speakers, while others speak a different language at home and, despite having had all 
their education in the United States, are only marginally proficient in English. Whatever the reason 
for the problem, colleges and English as a Second Language (ESL) programs generally rely on 
standardized multiple choice tests to determine whether to accept these students, where to place 
them, and when they have satisfied exit criteria. The native English speakers are likely to be placed 
in remedial courses, whereas the U.S. -educated non-native speakers are usually steered into ESL 
classes intended for international students, whose needs are entirely different from theirs. 

The standardized tests, for which students may prepare by undergoing expensive coaching 
and memorizing lengthy word lists, are unrelated to the actual language tasks required in college 
and have only low to moderate correlations with college grades. They also underpredict success for 
minority students, women, and ESL students. The composite numbers reported as test results give 
no infonnation about the nature of the strengths and weaknesses of the individual. 

The goals of this project were to develop an assessment of students’ academic language 
proficiency and a curriculum to improve their academic language skills. 
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Innovative Features 



The assessment and the curriculum developed for this project are both based on 
videotaped college lectures and excerpts from college textbooks. The curriculum focuses on 
the major language differences between under-prepared and well-prepared students. These 
differences were determined through detailed analyses of field tests of 700 college and high 
school students, representing a wide range of background and preparation levels. 

To create the pre- and posttests the project director first studied the nature of the language 
used in college courses, and defined the language tasks required in college. Project staff videotaped 
eight 30-minute lectures and selected one for the tests, while others became the basis for the 
curriculum. The test lecture focuses on a topic, theories of learning, that the students are not likely 
to have encountered before. The test lecture is difficult enough— in its proportion of academic 
vocabulary, speed of delivery, and sentence complexity— to avoid a ceiling effect in grading, and it 
is well organized, with cues for important topics and repetition of key points. 

The tests consist of the following elements: 

1 . Lecture notes. Students watch a lecture excerpt (with an introduction that includes 
an overview of the topic, key vocabulary, and instructions for note taking) and take notes. 
These are scored holistically for inclusion of the most important information and for evidence 
of such skills as using abbreviations and indicating relationships among topics. 

2. Short answer questions (based on the lecture). Scored for precision, vocabulary, 
grammar and spelling, this task indicates comprehension and requires students to synthesize 
information from the notes, using their own words. 

3. Dictation of lecture sentences. This exercise is scored for correctness of academic 
vocabulary and for word endings, which correlate with other indicators of academic language 
proficiency (whereas minor misspellings do not). 

4. Self assessment of academic vocabulary (reading and dictation). Field test results 
revealed this as a valid and reliable method for assessing vocabulary knowledge. Students rate 
their level of knowledge of each of 25 read or dictated words on a one-to-five scale. Students 
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are generally more confident about the words they read than the words they hear, perhaps 
because exposure to academic language in high school occurs more through reading than 
through oral use. This indicates that practice with lectures is important for students with 
academic language deficiencies. 

5. Sentence completion (reading). This exercise evaluates the ability to recognize the 
correct morpheme endings of common words. Students read simplified passages related to the 
lecture and for certain words select the correct ending that fits the sentence. 

6. Reading importance. In passages from textbooks related to— but not repetitive of— 
the lecture topic, students underline the twelve most important propositions. 

7. Notes from reading. After the underlining task, students take notes from an excerpt 
of the reading selection. The notes are scored for the presence of important information, 
evidence of note^taking skills and absence of verbatim copying. 

8. Short answer questions (based on the reading). The reading excerpt is collected, and 
students answer a general question using only their notes. This writing sample and the lecture 
writing sample are scored on a ten-point rubric for writing correctness and for content accuracy. 

After the tests are scored, each student receives an individual report which shows his or 
her strengths and weaknesses and, in the posttest, relative progress. There is no combined, 
overall or passing score, because this would result in the loss of specific information about 
students’ strengths and weaknesses. Normative information for interpreting test results is 
provided on the student score reports. 

The curriculum emphasizes academic vocabulary because the initial field tests revealed that 
its absence constitutes by far the most important barrier to the comprehension of lectures and text. 
Students learn prefixes, suffixes and roots that indicate meaning. They also learn the nominal, 
verbal, adjectival and adverbial forms of a word and maricers that indicate plural, past tense, or 
gerund forms. To help remember the meaning of new words, each is marked with an icon 
representing one of eleven semantic categories such as person characteristics; cause, effect and 
change; evaluation and description, and so on. Each lesson has a dictionary in which new words 
are defmed in non-academic language, and used in context. 



The lessons also provide instruction on note-taking, on recognizing the markers of 
lecture organization, and on reading strategies such as identifying important infonuation and 
analyzing complex sentences. Because many students who need this kind of assistance are also 
the first in their families to attend college, the curriculum includes information on academic 
culture. 

Evaluation and Project Impact 

The assessment and curriculum have been implemented at various levels at 19 sites, 
including high schools, community colleges, and universities. 

One group of community college students served as controls by taking the pretest and 
the posttest before and after their first two semesters of college with no exposure to the project 
curriculum. The results showed that merely attending college classes has no measurable effect 
on the academic language skills development of underprepared language-minority students and 
that practice on the tests alone, without benefit of the curriculum, does not increase scores. 
Academic language scores for these students showed a moderate negative correlation with the 
number of units dropped during the two semesters. In other words, the lower the academic 
language proficiency, the more units students withdrew from during the year. This suggests 
that low academic language skills are related to low persistence and low achievement for 
beginning college students. 

The longest use of the assessment and curriculum so far took place in a large, 
predominantly Hispanic, rural comprehensive high school with a 50 percent graduation rate. 
During 1996-97, 103 students took the pretest, and 84 the posttest. The results show that, 
after exposure to only three of the eight lessons— taught part-time over five months— the 
students made significant progress, especially in taking notes from lecture and reading and 
answering a reading question using their notes. There were also significant increases in 
self-ratings of vocabulary knowledge and small increases in other areas of the test. 



In a college remedial reading class, attendance and retention changed from extremely 
poor to perfect when the project curriculum was introduced, and students found the curriculum 
valuable and interesting. Most importantly, scores in the posttest showed significant increases 
in academic language skills. 

Results from an Upward Bound program site showed statistically significant increases 
in academic language test scores. Interviews with students at this site revealed positive 
changes in their attitude about and knowledge of academic language and academic culture. 
Students reported that the course helped them improve their grades and skills in their high 
school classes. 

Students enrolled in an anatomy and physiology course at one four-year institution were 
given academic language assessment tasks and the results were compared with their course 
grades. Correlations ranging from .35 to .50 were found between the various academic 
language tasks and the course grades. Students who received passing scores in the course had 
significantly higher academic language scores than those who received a D or F in the course. 
All students who completed the course had higher academic language scores than did students 
who withdrew from the class during the semester. 

In summary, in all contexts where the assessment and curriculum were used, students 
made significant improvements in their academic language proficiency and had positive 
attitudes toward the course. Evaluation evidence suggests that academic language proficiency 
is related to persistence and academic achievement in postsecondary education. 
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Lessons Learned 



Before the assessment and curriculum could be implemented, project staff had to 
overcome certain obstacles related to prevalent theories and traditional attitudes toward 
language learning and remedial instruction. 

Study skills teachers generally are not keenly aware of the actual requirements of 
regular college courses, and most base their teaching on the book and on their intuition— as 
opposed to factual information about what students need to know. In ESL classes, there is 
often even less awareness of what students need to succeed in college. Often, as ESL students 
become more orally fluent they are mainstreamed, without assessment of their writing and 
other academic proficiencies, which likely lag behind. 

The whole-language approach, which holds that literacy, like spoken language, 
develops simply through exposure, has often resulted in the neglect of specific instruction in 
reading, phonics, spelling and writing skills. Although many teachers are moving away from 
this approach, there is still resistance to such techniques as vocabulary instruction and 
skills-based approaches based on expository text. 

Academic language development is a long-term process. The curriculum and 
assessment created during this project aim to give students the tools to continue learning new 
vocabulary and overcoming their weaknesses long after the course is over. The project should 
continue to bear fruit as these students advance through the academic ranks. 

The results of this project confirm the notion that academic language is, in a sense, a second 
language for most students. On the pre- and posttests, the profiles of native speakers of English 
and of students classified as fully English proficient and limited English proficient were almost 
indistinguishable. In fact, in some tasks students with limited proficiency had slightly higher scores 
than the other groups— possibly because of better schooling in their first language. 

The curriculum is effective only with a teacher who understands the language needs of 
students and who is enthusiastic and effective. A teacher who does not engage and motivate 
students, showing them the relevance of academic language tasks, will not see much improvement. 



Project Continuation and Dissemination 



The curriculum continues to be disseminated and its effectiveness documented based on 
teacher and student reactions. The materials have been used by, among others, several 
university Upward Bound programs and at sites in several school districts. 

The project received a new grant under FIPSE’s 1997 Comprehensive Program 
competition to disseminate and evaluate the program’s effectiveness more widely. The new 
evaluation efforts include looking for long-term effects on students. At some sites the course 
may be expanded to a two-semester sequence. 



Available Information 

Further information may be obtained from: 
Phyllis A. Kuehn 

Department of Educational Research 
California State University, Fresno 
Fresno, CA 93740-8025 
Telephone: 559-278-0323 
e-mail: phyllis-kuehn@CSUfresno.edu 



III. IMPROVING TEACHING AND LEARNING 
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AMERICAN ASSOCIATION FOR HIGHER EDUCATION 
Forum on Faculty Roles and Rewards 



Purpose 



It is difficult to think of a way to improve learning that does not have the faculty at its 
core. In view of this, the American Association for Higher Education (AAHE) established the 
Forum on Faculty Roles & Rewards to begin a national dialogue on strategies to engage 
faculty more fully in their work. 

The dialogue began at a time when colleges and universities faced diminishing 
resources, anticipated mass retirements and subsequent faculty shortages in many fields, and 
felt intensified public demands for increased access and accountability. The threat of outside 
controls on such issues as student testing, faculty workloads, and fiscal management loomed 
larger than ever. 

While these troubles darkened the academic skies, however, they also created an ideal 
climate for the forum; faculty and administrators, disciplinary associations, and other higher 
education entities joined in the dialogue with an intensity bom of the conviction that the 
essence of academic life was at stake. 

Innovative Features 



The fomm was proposed in 1991 to address an emerging national issue: colleges and 
universities across the country were reexamining faculty expectations, evaluation and rewards, 
and legislators and boards of regents were beginning to raise serious questions about faculty 
workload and productivity. The mismatch between the fundamental missions of many 
institutions— particularly the commitment to quality undergraduate teaching— and the rewards 
being provided to faculty was of primary concern. The fomm set out to provide direction and 
practical guidance for campuses, especially those with complex missions, interested in 



reexamining faculty pribrities: The forum sought to address the needs of institutions that wanted to 
expand the notion of scholarly work beyond the traditional- research-focused model to include teaching, 
research, and service. The forum also sought to provide additional career-path options to new faculty 
and to face the issues presented by the impending generational changing of tlie guard among the faculty. 

But the forum did not simply hope to emphasize the need to reward fine teachers with 
external perquisites such as salaries and tenure. Its real intention was to find ways of integrating 
teaching, research and service so that faculty would receive internal rewards as well and would reap 
the sense of personal engagement and fiilftllment that is the only guarantee of lasting quality. 

Concerns about evaluating and rewarding teaching had been growing since the 1970s. 
The foruni, however, envisaged a national conversation around fresh formulations of these 
problems. Thus,, for example, it delennined to reexamine the old scholar-researcher model not 
just because more attention must be paid to teaching, but because the model has proven 
unnecessarily unidimensional. By rewarding only a narrow range of behaviors, the scholar- 
researcher model cheats academia of some of its most creative and talented contributors. It 
ignores the “ways of knowing” that expert practitioners develop over a lifetime, for instance, 
or the special talents required not only to create knowledge but to present it in a meaningful 
manner to audiences both in and outside the university. 

To stimulate and sustain the national conversation on faculty work the forum relied on 
tools that had served AAHE well in the past: annual conferences, publications, and the 
services of a clearinghouse of information. 

The conferences were designed to attract the academic leadership (i.e., the chief academic 
officer and the leader of the faculty governance body) of institutions whose faculty were expected to 
function as both teachers and scholars. To influence perceptions of the seriousness and importance 
of the forum’s work, the conferences featured nationally recognized authorities. 

In addition to numerous articles in AAHE publications, the project yielded three 
monographs: Jon Wergin’s The Collaborative Department, on departmental responsibilities in 
defining and rewarding faculty work; Ernest Lynton’s Making the Case for Professional Service, 
on the need for rethinking the service component of faculty duties; and Robert Diamond and 



Bronwyn Adam, eds., The Disciplines Speak, on the definition of scholarship. AAHE has 
recently published a companion volume to the Lynton monograph, Making Outreach Visible: A 
Guide to Documenting Professional Service and Outreach, by Amy Driscoll and Ernest A. 
Lynton, and discussions are underway regarding the publication of follow-up monographs to 
both the Wergin and Diamond and Adam pubUcatiGns. 

Finally, the forum built a clearinghouse that includes campus documents and articles 
from the press and answers hundreds of requests annually. Its staff are in wide demand as 
consultants on campuses where the dialogue on roles and rewards is being translated into 
practice. 

Evaluation and Project Impact 

The fonim’s timeliness as well as the quality of the conversation it instigated earned it 
an enthusiastic response in the higher education community. In fact, the problems that the 
project encountered had mostly to do with the volume of the response. The annual 
conferences attracted many more participants than anticipated— the fourth, one reached the 
1 ,000-registrant mark— and many institutions sent teams of faculty to each one. Over three 
years, more than 800 requests came in to the clearinghouse for resources, speakers and 
information. 

Some institutions that sent representatives to the conferences, such as the University of 
Wisconsin, have since started their own meetings on faculty roles and rewards. Others, such 
as the University System of Georgia, have integrated forum concerns into their planning 
documents. Forum topics and conclusions also inform, the pTograms of bodies such as the 
American Association of Schools of Business, the American Association of Medical; Colleges, 
and the Council of Independent Colleges. 

As the forum continues with a second three-year grant from FIPSE (see below) and its 
conclusions are applied on campuses across the nation, it will gradually become possible to 
Judge its concrete effect on the lives of faculty and on their students’ learning. 



Lessons Learned 



The fonim’s creators were surprised to find that the group they had expected to be the 
most recalcitrant to their ideas— academic administrators— was in fact the most eager to 
embrace them. Well acquainted with external pressures for accountability, budgetary realities 
and a shrinking faculty market in many fields, deans and provosts readily grasp the need for 
change. 

What resistance there is lies at the discipline and department level, which most faculty 
consider their professional “home.” Recognizing the importance of making progress in this 
area, during the second three-year project the forum is focusing on the disciplines and the 
departments. 

With regard to affecting the views of individual faculty, the forum’s organizers quickly 
realized that having institutional teams attend the conferences allows faculty to exchange ideas 
with each other and generates a level of enthusiasm that administrators would have difficulty 
creating on campus. 

During the course of the project it became apparent that one of the main uses of the 
forum is as a link among campuses that are trying out new ideas in relative isolation. It also 
became clear that although it was desirable for the forum to expand to include a variety of 
institutional types (and at present it is doing just that— see below), the nature and needs of 
two-year colleges call for a separate discussion of roles and rewards for their faculties. 

Finally, the creators of the forum concluded that the concept of “service” is in need of 
revision. The higher education community must come up with a broader, more flexible 
definition— one that will encompass the many ways in which faculty members can engage in 
their professional as well as their social communities. 
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Project Continuation 



With a second grant from FIPSE, the forum has nearly completed its second three-year 
period, focusing on disciplines and departments in the hope of achieving a direct impact on 
faculty. It has broadened its appeal to include “New American Colleges”— former liberal arts 
colleges with newly-diversified missions— and professional schools. The forum is expected to 
become self-sufficient at the close of the grant period. 

Dissemination and Recognition 

Two additional projects originated from the forum’s agenda and are in progress. 

“From Idea to Prototype; The Peer Review of Teaching” continues the examination of 
teaching portfolios among other strategies and has obtained funding from the Pew and Hewlett 
foundations. “New Pathways: Faculty Careers for the Twenty First Century” is expected to 
yield 14 working papers by its completion, on such topics as career transitions for senior 
faculty, and reconciling work and family. 

Available Information 



Further information may be obtained from: 

R. Eugene Rice 

Forum on Faculty Roles & Rewards 
American Association for Higher Education 
One Dupont Circle, Suite 360 
Washington, DC 20036-1110 
Telephone: 202-293-6440 ext. 37 
e-mail: ashaffrr@aahe.org 
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COLD SPRING HARBOR LABORATORY 
A Training Program for Small College Faculty in DNA Manipulation and Human 

Genetics 



Purpose 



Medicine, law, religion, civic life, family life, education— there is hardly an area of 
contemporary existence that has not been touched by advances in molecular biology and by the 
Human Genome Project in particular. As we enter the next century, this impact will continue 
to grow. Not only will today’s students be affected personally by applications of molecular 
genetics, but those entering the workforce in agriculture, medicine, manufacturing and law 
will need a basic knowledge of DNA science to exercise their professions. 

The single most likely place for students to at least begin to gain this knowledge is in 
introductory biology courses. Crucial for biology majors, these courses may be even more 
important for non-majors, because they are often the last exposure to the life sciences for these 
students. The nationwide Coalition for Education in the Life Sciences is especially concerned 
that students in these courses experience a worthwhile laboratory experience— something 
difficult to achieve in view of the large enrollments in these classes and the nearly universal 
funding shortages. 

Faculty in charge of these courses are feeling pressure to improve the laboratories— 
specifically, to introduce laboratories in molecular genetics— from an unexpected source: their 
students, many of whom received this kind of instruction in high school. The Educational Testing 
Service now mandates DNA restriction analysis and bacterial transfonnation experiments in the 
advanced placement biology curriculum. It is reasonable to estimate that as many as 75,000 
precollege students are exposed to laboratories in molecular genetics every year. 

Responding to these demands poses special problems, however, for faculty at institutions 
where they are expected to devote themselves to teaching. Over the years, they may have lost 
touch with current research and may lack the confidence to develop appropriate experiments, 
especially given the scarcity of laboratory equipment and resources at many such institutions. 
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The ultimate goal of this project was to introduce exercises in basic gene manipulation 
in first- and second-year college courses, thereby imparting “genetic literacy” to students and 
stimulating their interest in the life sciences. To achieve this, the project intended to give the 
teachers of these courses hands-on experience in molecular genetics through a series of 
laboratories focused on a simple prokaryotic system designed for large numbers of students. 

Cold Spring Harbor Laboratories’ DNA Learning Center was particularly qualified to carry 
out this task. Not only was the Laboratories’ director, James Watson, co-discoverer of the 
structure of DNA and founding director of the National Center for Human Genome Research, but 
many gene manipulation techniques basic to the genome project were developed at the laboratories, 
which also have a long and distinguished history of involvement in biology education. 

Innovative Features 

In the years Just before the project. Cold Spring Harbor Laboratories’ DNA Learning 
Center, the nation’s first institution devoted exclusively to public genetics education, 
developed and tested two basic molecular genetics curricula. The first, “DNA Science, A 
First Course in Recombinant DNA Technology,” is intended for advanced high school students 
and stops short of the DNA hybridization techniques needed to isolate a single gene from a 
library of genes. The second, “Laboratory DNA Science, An Introduction to Recombinant 
DNA Techniques and Methods of Genome Analysis,” features more advanced experiments 
related to the Human Genome Project, allowing for the construction and analysis of gene 
libraries. Together, the curricula can take a college freshman or sophomore with no previous 
knowledge of the subject to a comprehensive understanding of modem methods for isolating 
and analyzing genes. 

The curriculum’s laboratories use E. coli and its endogenous virus, lambda, to illustrate the 
techniques used in the analysis of eukaryotic genomes. Because of the bacterium’s fast generation 
time, small genome, and ease of manipulation, it can be used to introduce sophisticated molecular 
techniques without the technical and regulatory complexities of working with human DNA. 



The laboratories’ designers sought to modify current research protocols to minimize 
expense and ensure safety and reproducibility. And they strove to preserve the integrity of the 
research methods involved so that students going on to more advanced situations would not 
need to relearn them. Adapting research techniques to the teaching laboratory was no easy 
matter. For example, to simplify the interpretation of experiment results, DNA Learning 
Center staff developed three new plasmids which transforin well, are highly amplified in 
E. coll, and give consistent yields in plasmid preparations. These are probably the only DNA 
molecules specifically engineered for educational purposes. 

Because the new protocols are biochemically complex and require novel reagents, the DNA 
Learning Center collaborated with a number of corporations to redesign research-grade kits for 
education. They also issued a license to a commercial supplier to market two polymerase chain 
reaction kits— stand-alone experiments that employ simplified protocols and inexpensive substitutes 
for equipment, reduce preparation time, and are straightforward to use. One kit allows students to 
amplify a segment of their own DNA using a polymerase chain reaction. 

The faculty development workshops that project staff created to ensure the broad 
dissemination of the curriculum were intended to put DNA manipulation technology within the 
reach of any motivated college instmctor. The workshops built on the introduction to DNA 
provided by most standard survey texts. The laboratories presupposed no prior knowledge of 
molecular technology, and they proceeded cumulatively, repeatedly bringing major techniques 
into play to allow participants to gain confidence in their use. 

Training was offered in two sets of intensive ten-day summer workshops located at major . 
universities. The workshops were held at Howard University and the University of Chicago the 
first summer and at Columbia University and California State University, Northridge, the second. 
The host universities were chosen because they were accessible to faculty from a large number of 
institutions and because their prestige would attract applicants and underscore the rigor of the 
experience. Winter follow-ups were held in the same locations. Workshop staff consisted of a 
senior instructor, a coordinator, and a college intern who served as a laboratory aide. 
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The workshops featured 22 experiments. They began with an introduction of basic 
techniques of microbial culture and proceeded to agarose gel electrophoresis, DNA restriction 
and ligation, and plasmid isolation. These techniques were then extended to gene library 
construction and screening, Southern hybridization, polymerase chain reaction, and human 
DNA fingerprinting. 

In addition, lectures related the laboratories to the Human Genome Project and to 
genetic medicine. Local academic and industry researchers gave seminars on applications of 
the laboratory techniques presented. And participants discussed the ethical, social and personal 
issues raised by the ability to manipulate the human genome, as well as the financial, 
scheduling and safety issues involved in teaching the new laboratories. 

The follow-up workshop was intended to reinforce faculty confidence and to explore in 
more detail the practical aspects of laboratory-based teaching in genetics. It included three 
entry-level experiments that used elements of the laboratories presented during the summer: 
E-Z DNA recombination, restriction mapping, and forensic plasmid identification. 

Evaluation and Project Impact 

The workshops attracted a total of 84 participants. 75 percent of them were white, 16 
percent black, 5 percent Asian, and 4 percent Hispanic. Sixty-three percent were male and 37 
percent female, reflecting the makeup of the applicant pool. Forty-three percent came from 
colleges with under 5,000 students, and four out of ten came from institutions half or more of 
whose students were minority or disadvantaged. Two out of ten came from institutions where 
20 percent to 49 percent of the students were minority or disadvantaged. Two-thirds of the 
home institutions were public; one-third were private. 

Participants were surveyed before and after the workshop, and again 17 months after the 
experience. The surveys generated a 79 percent response which revealed that the curriculum had 
been well received— due in part to the experiments’ 80 to 100 percent success rate, one result of 
their thorough prior testing. Faculty suggested modifications to the schedule to reduce confusion 
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caused by overlapping experiments. They were particularly pleased with the seminars and 
discussions on the social, ethical and legal repercussions of genetic technology and the history 
of the eugenics movement in the United States. 

The majority of participants reported that they had incorporated two to three 
laboratories into their teaching in the year after the workshop and planned to incorporate more 
as equipment and reagents became available. Entry-level laboratories (bacterial transformation 
and DNA restriction analysis) were the first to be used. Half of the participants had made 
presentations on workshop topics to their colleagues, ten percent had made presentations at 
professional meetings, and ten percent had led teacher training workshops. 

Predictably, faculty stated that the main constraint to teaching laboratory-based 
molecular genetics was the lack of equipment. But here again there may be cause for 
optimism-half of the respondents attempted to obtain funds or equipment donations for the 
new laboratories and succeeded to the extent of matching FIPSE per-participant costs at a 
seven-to-one ratio. 

The DNA Leaniing Center continues to track faculty in the original and following 
workshops in ways comparable to the tracking of 2,000 high school teachers who received 
professional training from the center. The resulting database should yield a useful picture of 
genetics instruction at the advanced high-school and beginning college level. 

Lessons Learned 



Perhaps because of the absence of direct participant support, applicant response to the 
workshops was less than half of the response to similar programs offered by Cold Spring 
Harbor Laboratories. Staff had geared the workshops strictly to teaching and had to carefully 
screen out the many applicants who wanted to learn DNA techniques for their own research. 

In choosing the participants, staff selected not only for motivated applicants but for “motivated 
institutions” — those who showed a willingness to offer their faculty the kind of support they 
would need to integrate the workshop’s teachings into their own courses. 
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Project Continuation 



Due to lack of funding, the workshops are no longer offered. 

Dissemination 



Laboratory DNA Science: An Introduction to Recombinant DNA Techniques and 
Methods of Genome Analysis , by Mark V. Bloom, David A. Miklos, and Greg A. Freyer was 
published in 1995 and incorporates feedback from workshop participants. 

Available Information 



Further information may be obtained from: 

Mark Bloom 

DNA Learning Center 

One Bungtown Road 

Cold Spring Harbor, NY 11724 

Telephone: 516-367-7240 

e-mail : bloom@cshl . org 
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SETON HALL UNIVERSITY 

Assessing the Teaching and Student Learning Outcomes of the Katz/Henry 
Faculty Development Model 



Purpose 



The Katz/Henry model of faculty development, Partners in Learning (PIL), evolved 
during a decade-long collaboration between Joseph Katz and Mildred Henry which was 
supported by one of FIPSE’s first grants in the early 1970s. In the late 1980s PIL, financed 
by the New Jersey Department of Higher Education and administered by the Woodrow Wilson 
National Fellowship Foundation, was disseminated among public and private postsecondary 
institutions in New Jersey. Now PIL is housed in the New Jersey Institute for Collegiate 
Teaching and Learning, at Seton Hall University. Over the last decade, PIL programs have 
operated on 27 campuses in New Jersey and involved over 500 faculty. 

Fully described in Katz and Henry’s 1998 book, Turning Professors into Teachers, PIL 
is unique among faculty development methods in that it focuses on student learning and is 
based on classroom interactions. Concerned with process rather than goals, the model is 
faculty-run, non-judgmental, and encourages innovation and exploration. It seeks to 
reinvigorate faculty and alleviate the stresses and isolation of the classroom. 

Faculty pair off and exchange observer-observed roles in the classroom on a 
term-by-tenn basis. The faculty pairs meet frequently, interview students several times, and 
join other participating colleagues for general discussions of teaching-related matters. 

According to prior evaluations of Katz/Henry programs, faculty felt that they had made 
significant improvements in their teaching, but these self-reports could not be verified. The 
increasing concern with the quality of instruction and faculty development, combined with 
decreasing financial resources at most institutions, prompted the director of this project to try 
to find out whether there was in fact a link between the PIL approach to faculty development 
and student learning. 
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Innovative Features 



Twenty-four faculty from four institutions—Rutgers University, Newark, Bergen 
Coinniunity College, the County College of Morris, and Kean College — participated in the 
assessment. Mostly Ph.D. holders, they came from all ranks and averaged 19 years of 
teaching experience, ranging from six to forty years. Some had considerable faculty 
development experience, while others had only recently become interested in improving their 
teaching. Still others appeared somewhat disinterested but were motivated by the stipend and 
release time offered to participants. 

Faculty were coupled in three interdisciplinary pairs per institution, typically a social 
scientist and a natural scientist. Each pair’s work revolved around a focal course~an 
introductory-level offering that was part of the observed instructor’s regular load. The 
partners interviewed a minimum of three students three or four times per semester, kept a 
journal of their observations, and attended monthly sessions at the New Jersey Institute for 
Collegiate Teaching and Learning to discuss their experiences and explore innovative 
pedagogies. 

Two of the three pairs on each campus were active in years one and two and fallow in 
three. The remaining pair (the control) was fallow in year one and active in years two and 
three. Data were collected on fallow pairs in year one to establish the baseline control and on 
fallow pairs in year three to determine the persistence of change. 

Students were surveyed with pre- and posttests regarding their preference for class 
structure and teaching style, their interactions with faculty and their experiences in the course, 
faculty teaching behaviors and their impact on learning, personal and intellectual changes 
resulting from the course, and progress in academic skills. The posttest also included an 
academic achievement question — graded anonymously by the course instructor — to reflect the 
students’ mastery of the subject matter. Responses to this question were controlled for level 
with the students’ GPA and college placement test scores. 
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Project staff observed faculty in the classroom a minimum of three times per semester. 
They looked for clarity, enthusiasm, organization, and pacing of the delivery, and interaction 
and rapport with students. Course syllabi were collected and analyzed, and each participant 
was interviewed. In addition, faculty completed two attitudinal surveys closely resembling the 
students’ posttest. 

Project Impact 

Most faculty believed that as a result of PIL they had increased the diversity of their 
teaching strategies— especially using more active learning approaches— and their awareness of 
their influence over students. They also stated that because of the program they understood 
better how students learn and improved their relationships with colleagues as well as their 
teaching. 

Students agreed that their instructors had changed, citing more and better interactions 
with them and higher academic expectations. But they reported perceiving no significant 
changes in in-class teaching behaviors or personal or intellectual changes as a result of being in 
the course. There were no discernible patterns of increased academic achievement in the 
students of participating faculty, even when controlling for ability. 

Using infoniial interactions with faculty, the journals, and the exit interviews, staff 
plotted each faculty participant on a grid illustrating the perceived degree of change in teaching 
and the amount of personal reflection about the process. Three clusters emerged— those at the 
low end of the scale, labeled “resisters,” those in the middle of the grid, the largest number, 
labeled “moderate changers,” and those at the upper end of the scales, labeled “converts.” 
Staff then used the students’ and faculty’s self-reports to test these designations. On the 
whole, the typology was not supported: distinguishing between faculty types based on the 
students’ responses or the faculty’s self reports was, with few exceptions, impossible. 
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Lessons Learned 



Project staff speculate that the inconclusive results of the assessment may be due to the 
failure to select the appropriate outcomes. Faculty enter a project such as this one with highly 
individual and idiosyncratic goals, and these goals are not likely to be reflected in the few 
general outcomes selected for assessment. Another important factor is the nature of the 
population— the faculty who chose to undertake PIL were for the most part committed to good 
teaching and thus less likely to show dramatic growth than a randomly-selected group. 

A number of factors made implementation difficult. To maintain PIL’s non-evaliiative 
tone, it is important that faculty set their own agenda and rules, with minimal intervention 
from the project coordinator. In an assessment situation, however, this makes control of the 
“treatment” all but impossible. During the project, events such as sabbaticals, illness, changes 
in teaching load, and schedule and personality conflicts conspired against consistency. 

It was challenging to keep faculty motivated to participate and furnish data over three 
years. But it was even more challenging to collect academic data from students, who were not 
especially interested in providing it and received no encouragement from their instructors. 
Faculty were reluctant to take up course time with project-related activities, and, despite the 
fact that students signed a consent form, faculty members said they felt that it was unethical to 
ask them to provide academic data that were not relevant to the course. 

The project generated massive amounts of data from different sources and in different 
forms. These were hard to analyze in a meaningful way, especially when quantitative and 
qualitative data differed on the same issue. 
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Available Information 



Further information may be obtained from: 
Martin Finkelstein 

College of Education and Human Services 
Seton Hall University 
South Orange, NJ 07079 
Telephone: 973-275-2056 
e-mail: finkelma@shu.edu 
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SOUTHWEST TEXAS STATE UNIVERSITY 
Interactive Video Listening Comprehension in Foreign Language Instruction: 

Development and Evaluation 



Purpose 

According to recent research, the ability to understand a foreign language in its spoken 
fomi leads to proficiency in speaking and writing that language. Listening comprehension is 
now widely believed to be the primary mechanism by which learners acquire competency in 
foreign languages. 

During the last two decades, computer-assisted language learning has played an 
increasing role in foreign language instruction. Especially where listening comprehension is 
concerned, highly individualized interactive videodisc programs offer important pedagogical 
advantages over traditional audio approaches. The computer— a patient tutor ever at the 
student’s beck and call— pennits instant and repeated access to precisely defined video 
sequences that focus attention on specific linguistic features and supports the student’s progress 
by comprehension checks and a wide array of other pedagogical features. 

Such programs are rare, however, and on the whole computer instruction has failed to 
reach its full potential in the foreign language classroom. Most generic computer programs 
offer little more than simple drills that match neither the instructor’s nor the students’ needs. 
Unfortunately, many foreign language faculty lack both the expertise to write computer 
programs and the time in which to learn to do so. 

Authoring systems, by eliminating the need to program the computer, allow faculty to 
develop multimedia lessons which focus on listening comprehension and match their— and their 
students’— specific requirements. This was the goal that project faculty set for themselves 
when they created Libra. 
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Innovative Features 



Available in both Macintosh and IBM-compatible versions, Libra consists of a series of 
templates for basic expository displays, various question fonnats with feedback, and help 
features such as scripts and dictionaries. It allows the creation of multimedia controls, dialog 
boxes, hyperactive texts, response-sensitive lesson controls, links to graphics, digitized audio 
and video, and World Wide Web documents. At all times during the development of a lesson 
the author sees exactly what the student will see. Each authoring tool is designed to enable 
faculty to assemble lessons which model appropriate listening comprehension strategies and 
guide students in their use. 

Research shows that presenting certain kinds of information (known as “advance 
organizers”) concerning the story that students are about to view helps them to construct a 
meaningful mental representation of the text. Thus, Libra’s text tools allow displays which 
introduce the story and its setting. Schematic displays of pictorial cues reveal the narrative 
structure of the text and help students to relate the text’s components to each other to form a 
coherent overview of the story. Icon buttons representing the events in the text can be 
arranged in patterns reflecting the text’s narrative structure. And authors can construct text 
maps which portray linear plot progressions, causal chains, and other narrative structures. 

Effective advance organizers underscore key information in the story and prepare 
students for expressions that may be difficult to understand. Libra can create hyperactive text 
displays linked to explanatory devices such as videodisc events or digitized audio files. 

Students click the underlined expressions and hear them as they are spoken in the video scene, 
replaying them as needed. 

According to the findings of linguistic research, individuals skilled in comprehension 
use flexible strategies at several levels of text processing. Libra allows faculty to direct 
students both to developing their general understanding of the text and to processing specific 
information from it through five kinds of comprehension checks that verify students’ 
understanding of general as well as specific information. 
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Libra offers a number of help features. For students who have difficulty identifying 
relevant sentence constituents, the system’s ancillary stacks contain templates to develop displays 
which provide a progressive buildup of a complex sentence, accent its important components, and 
show their interrelationships. Hyperlinked words or expressions offer additional help displays. 

The ancillary stacks also allow creation of a help display equipped with a set of video 
buttons to delimit videodisc segments which contain important information. Finally, Libra’s 
script-building tools can be used to develop written scripts of the video text. 

Upon completion of the authoring system, project faculty produced and piloted three 
sets of multimedia lessons based on authentic video materials in French, Gennan and Spanish. 
Faculty at three other institutions produced and piloted additional lessons in these languages. 

Evaluation and Project Impact 

Project faculty evaluated lessons developed with Libra as well as the authoring system itself. 

Data about the impact of Libra on learning came from French and Spanish lessons 
developed at four institutions, ranging from a highly selective private university to an 
open-admissions community college. Data were also collected on German lessons developed 
at Southwest Texas State, but because their design did not follow the suggested guidelines, the 
results did not reach statistical significance. 

In one study, the performance of students using Libra-authored lessons was compared to 
that of peers using more traditional video and computer pedagogies. The students in the 
experimental groups understood significantly more of the video material, wrote higher quality 
free-recall protocols, and made fewer interpretation errors than their peers in the control groups. 

In another study, the experimental group worked with a Libra-authored interactive 
videodisc while the control group viewed a linear videotape of the same narrative, supplemented 
by a paper replication of the computer instruction material used by the experimental group. In 
the last three of a series of five quizzes on listening comprehension, the experimental group 
outperfonned the control group at a statistically significant level. The absence of a statistically 
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significant difference between the results of the first two quizzes may have been due to some 
students in the experimental group not being aware initially of all the pedagogical devices in 
the Libra-authored lessons. 

In a similar test at another institution, the experimental group consistently outperfomied 
the controls. Two other tests of improvement of listening comprehension (conducted without 
controls) showed that comprehension increased by 73 percent and 86 percent after using the 
Libra lessons. 

All students surveyed stated that they enjoyed using the Libra-authored lessons, and 
more than three-quarters believed that the lessons had helped them to improve their listening 
comprehension. A pre-post comparison of attitudes toward using the technology shows a 
strong positive shift, with over half the students selecting the highest rating on a five-point 
scale and describing themselves as “enthusiastic” about the lessons after having used them. 
Students using linear videotape showed much less change in their attitude. 

The authoring system itself was evaluated by means of a written questionnaire 
administered to 60 foreign language educators trained by project staff. The respondents 
ranged from graduate faculty to middle school teachers, and from computer novices to expert 
users. Libra received highly favorable ratings, especially on questions regarding ease of 
learning and using the system and the degree to which it met the individual instructor’s needs 
(about eight on a ten-point scale). 

Lessons Learned 



Initially project staff estimated that it would take ten hours a week over a year and a 
half to develop Libra. In fact, they spent a minimum of 15 hours weekly discussing, 
expanding and improving the system. But they believe that this careful work in the early 
stages was crucial to the system’s success. 
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Project Continuation 



Libra continues to be used at Southwest Texas State, and project faculty have 
developed over 50 lessons in French but few in German or Spanish. 



Dissemination and Recognition 

An article entitled “The Instructional Basis of Libra” appeared in the Journal of Language 
Learning Technologies. Project staff demonstrated the system at meetings of the American Council 
on the Teaching of Foreign Languages, the Computer Assisted Language Instruction Consortium, 
the South Central Association of Learning Laboratories, and the South Central Modem Language 
Association. They continue to offer authoring workshops to faculty at academic institutions and 
professional meetings across the country. Two hundred-fifty faculty have purchased libra. 

In 1994 the university won a second grant from FIPSE to disseminate Libra to 
institutions across the United States. Several faculty groups participating in the dissemination 
workshops obtained up to $70,000 in local funding to continue their work. 

Recently the project obtained a third grant of $430,000 from the U.S. Department of 
Education’s International Education Office to continue this work and extend it to reading 
comprehension. 

Available Information 

Additional information is available from: 

Robert Fischer 

Department of Modem Languages 
Southwest Texas State University 
601 University Drive 
San Marcos, TX 78666 
Telephone: 512-245-2360 
e-mail : fr02@swt . edu 
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TUFTS UNIVERSITY 

Detailed Evaluation of a Novel Approach to Curricular Software 



Purpose 

Exploding numbers of underprepared students and shrinking budgets are only some of 
the factors that make teaching on American campuses today increasingly difficult. Under these 
conditions, it is only natural for faculty and administrators to turn to the computer for the 
solution to problems ranging from cost effectiveness to remediation. 

Educational software holds immense promise for solving these dilemmas and improving 
teaching and learning, but it is difficult to design, requires costly equipment, and demands 
considerable adjustment in the way that faculty teach. Given the weight of these investments and 
the growing popularity of computers in the classroom, it is imperative that new software receive a 
thorough and uncompromising evaluation. 

From 1987 to 1990, with FIPSE funding, the Curricular Software Studio at Tufts developed 
ConStatS, a software package to help introductory statistics students to assume an active, 
experimental style of learning. The designers of ConStatS believed that developing this learning 
style was essential to the acquisition of a deep conceptual understanding of the subject. Every 
ConStatS program is divided into many screens, each of which confronts the student with a small 
number of closely related decisions. The studenf s choices lead down different pathways, some of 
which loop into each other. At any point, the student can back up along a pathway, or use the 
WHY and HELP buttons available on every screen to obtain additional information. 

Although, during the original project, formative evaluation showed that ConStatS was 
free of bugs and students felt comfortable using it, its creators wanted to find out whether the 
experimental approach that the software fosters made a difference in learning. They also 
wanted to find out how students used the software and which portions of it worked with what 
type of students, especially underprepared ones. Finally, they wanted to provide an example 
of good software evaluation, a model that would focus on comprehension and retention rather 
than on student attitudes and speed of learning, as much technology assessment does. 
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Innovative Features 



Several factors made it imperative that the evaluation include a number of sites. 

Project staff wanted to find out whether ConStatS would help students in different courses, 
different departments and different institutions to learn statistics. Besides, if only one or two 
sites were used, factors unique to the course or the institution could overly influence the 
results, a common defect of technology evaluations. 

Accordingly, staff recruited faculty at the University of New Hampshire, Gallaudet 
University, Colorado College, and Bowdoin, in addition to Tufts, to use ConStatS as they saw fit. 
This flexibility in how the software was used made faculty at all five sites eager to cooperate. 
Unfortunately, it was not possible to recruit a community college, perhaps because these institutions 
offer fewer statistics courses and have less technology. Overall, 20 introductory statistics courses 
participated in the project. Sixteen used ConStatS and four did not. 

Because faculty use ConStatS in many different ways (Tufts alone has eight 
introductory statistics courses in various departments, all of which employ ConStatS 
differently), the evaluation needed to include a record of how the program was used. Project 
staff used a multivariate assessment measure to capture the effectiveness of each part of the 
program and performed detailed testing and extensive monitoring of implementation. 

During the first year of the project, staff developed a pretest of the skills necessary for 
students to use ConStatS and a detailed test of conceptual understanding that assessed every 
part of the program, to be taken by students in the twenty participating courses. They also 
designed a feature in the program for recording how each student used the software. The test 
of conceptual understanding required a detailed analysis of over 200 ConStatS screens, which 
yielded over 1,000 comprehension points. Because it was impossible to test all the points, 
they were grouped into clusters, which in turn yielded 103 concepts. For each concept, a 
question testing understanding was formulated and then reviewed by two outside statistical 
experts and teachers of quantitative methods. 
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Designing the feature for recording interaction required describing the kinds of 
interactions offered by ConStatS and linking them to the educational purpose that motivated 
each one—such as executing an experiment, asking for help, or studying a result. The next 
step involved engineering a software tracing function that would capture each interaction, 
interpret it, and include it in a database. This permitted project staff to account for differences 
in performance based on differences in use. 

All participating faculty were given a one-day workshop before the evaluation began. 
Project staff communicated biweekly with all the sites, and made visits to install the program 
and perfonn related tasks. In the second and third years of the project, students in the 20 
participating classes took the test of conceptual understanding, which consisted of questions on 
103 statistics concepts taught both in the software and in the classes that did not use the 
software. 

Project Impact 

The evaluation results strongly suggest that ConStatS makes a big difference—students 
using ConStatS did better on 94 out of the 103 questions testing understanding of statistical 
concepts. These results do not appear to be due to increased learning time, because at least 
two courses substituted a computer laboratory for a class. Nor are they tied to a particular 
way of integrating ConStatS into the curriculum, because the results were comparable among 
classes with different approaches. 

Students with poor mathematics backgrounds achieved fewer gains than better-prepared 
students. A ten question pretest examining basic mathematics proficiency (middle- and 
high-school levels) predicted accurately which students would gain most from ConStatS at all 
sites, perhaps because low scores on the test were symptomatic of serious academic deficits. It 
is possible, however, that if students had been tutored in fractions and ratios (the two most 
commonly missed items on the pretest) they would have shown greater gains from ConStatS. 
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The trace data enabled staff to connect differences in class scores and software use. 
Some classes made more extensive use of certain portions of the program than others. Also, 
trace data allowed the linkage of student performance to the different ways in which individual 
students used the software. In general, students who tended to stay in one spot in the program 
or moved slowly were less likely to be engaged by ConStatS and learned less from it. 

The evaluation confirmed the project staffs impression that most students leave 
introductory statistics with little conceptual understanding of the material. Students who did 
not use ConStatS averaged only 31 percent on the test of conceptual understanding. Even 
though scores were substantially higher in classes that used ConStatS, students were still not 
learning statistical concepts as well as they should have. 

Lessons Learned 



Project staff had expected the evaluation to reveal ConStatS to be an unqualified 
success. Instead they learned that, although the software is undoubtedly useful, students 
failed, for unknown reasons, to learn a number of concepts. Some of the handsomest 
illustrations in the program, the ones that received the most acclaim at conferences, did not 
promote learning. It is difficult, staff found out, to anticipate how students will learn from a 
given illustration. 

The hope that all students would engage in active experimental learning with the help 
of well-designed software was not completely fulfilled, because students with learning deficits 
showed only a marginal gain. For students whose mathematics background was appropriate 
for college-level work, however, ConStatS was helpful. 

Project Continuation 



The software continues to be used at Tufts in psychology, economics, biology and civil 
engineering courses. 
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Dissemination and Recognition 



ConStatS has been the subject of many conference presentations and a number of 
articles in refereed Journals. It was published by Prentice-Hall, Inc., in 1996. 



Available Information 



For further information, contact: 

Richard Chechile 

Psychology Department 

Tufts University 

Medford, MA 02155 

Telephone: 617-627-3765 

e-mail: RCHECHIL@em erald . tufts . edu 
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WEST VIRGINIA UNIVERSITY 

Development of a Computer System to Educate Students to Evaluate and Interpret 

Published Drug Studies 



Purpose 

Despite the efforts of peer reviewers and editorial boards, it is not unusual for errors in 
research design and inconsistencies in reporting to surface in professional pharmaceutical and 
medical journals. If patients are to receive appropriate treatment, it is essential that medical 
professionals be capable of thoroughly and accurately interpreting the primary literature on drugs. 

Nevertheless, the education of students in the techniques of appraising drug studies has 
typically received short shrift in many health sciences curricula. Often, the checklists, scoring 
systems and algorithms available to assist medical personnel to evaluate clinical drug studies 
are brief and do not adequately explain the concepts involved. 

Faced with the need to teach students to analyze drug literature, project faculty 
reasoned that it would make sense to use computer assisted instruction (CAI), because the 
computer allows for numerous examples and levels of explanation that students can access and 
review as needed. Faculty also believed that CAI would promote active learning in a 
curriculum that students mastered mostly by listening to lectures and memorizing facts. 

Innovative Features 



Faculty developed a self-contained interactive CAI program, entitled “Evaluation of 
Clinical Drug Studies,” initially to be used in conjunction with a one-credit course. Introduction to 
Drug Literature. However, the program has also been used as an independent two-credit course. 
The program teaches students to analyze and evaluate aU aspects of published clinical drug studies, 
with a focus on drug efficacy. It can be used to replace class lectures. The program features sound 
as well as graphics and includes presentation of material, examples, questions, and responses. 
Students can choose to review prior lessons or progress to new ones. 



The project started with a three-day training offered by the manufacturer of Authorware 
Professional 7 (Macromedia, San Francisco, Calif.), the software chosen for the program. 

This user-friendly software only requires knowledge of computer basics and allows the 
incorporation of sound, graphics and animation. 

Because the “Evaluation of Clinical Drug Studies” material had not been taught before, 
the team of faculty developers first had to outline the course and prepare the text. Three 
consultants and a senior pharmacy student reviewed the CAI materials at different stages in 
their development. 

The program consists of 16 sections and subsections on topics such as experimental design, 
study settings and patient factors, measurements, statistical concepts and analyses, and data 
handling. Students were given basic instruction on the computer and then assigned in groups of 
two or three to a computer. Each group received a different published clinical drug efficacy study 
to review as part of the course. Six exercises during the semester asked questions about the study 
and helped faculty evaluate students’ comprehension of the program. Students were expected to 
cover one section of the program per week on their own. About 15 minutes a week of class time 
were initially allotted for questions and discussion of the program. 

Evaluation and Project Impact 

A pretest and a posttest were administered to a control group and a test group of 
students. 

The control group consisted of 77 pharmacy students taking Introduction to Drug 
Literature the year before CAI was introduced. These students had had no formal instruction 
in drug study evaluation. The pretest was designed to measure their understanding of 
important characteristics of clinical drug studies and their ability to analyze and evaluate them. 
A similar posttest was administered at the end of the course. 

The test group consisted of 76 students enrolled in Introduction to Drug Literature with the 
CAI component. The pre- and posttests were almost identical to those given to the control group. 
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Both groups had similar age and GPA profiles. Slightly more students in the control 
group had prior experience with statistics or research methods than those in the test group. 
Although this was expected to yield better pretest scores on the part of the control group, the 
test group’s scores were higher by two percentage points. 

The mean posttest scores of the test group showed a significant increase over the pretest 
scores— from 43.6 percent to 77.4 percent. The posttestscores of the control group 
unexpectedly declined. This proved to be the result of anomalies in the administration of the 
test. When corrected for these anomalies, the resulting scores were quite similar— 43.8 
percent in the pretest versus 42.2 percent in the posttest. Similar tests administered to a later 
class of CAI users exceeded the first year’s findings: students scored an average of 42 percent 
on the pretest, and 83 percent on the posttest. 

Faculty had estimated that the CAI program would take about 15 hours to complete 
Students, however, turned out to be less assiduous than expected, spending a total of 11 hours 
on average, with a minimum of four hours and maximum of 25 hours. Almost all students 
skipped at least one section, and nine percent skipped more than half of the program. 

At first glance, only a weak correlation emerged between the total time spent on CAI 
and the posttest scores. However, there was a considerable difference between the time spent 
on the CAI program by students with the highest versus those with the lowest posttest scores. 
Those with scores of 90 percent and above spent on average at least ten hours more time on 
the CAI program than the students with the lowest scores. 

Student attitudes toward the program were measured by four surveys administered over one 
semester. Despite complaints that they did not receive enough credit given the additional load 
represented by the program, students generally felt that the information they were learning was 
new, of high quality, and geared to the appropriate level. They liked the CAI screen design and the 
way in which information was presented. They were almost equally divided on whether they 
preferred learning through lectures or through CAI. Despite the agreement among pharmacy 
educators about the need to learn to evaluate drug studies, some students felt that the program, 
although appropriate for physicians and researchers, was not necessary for them. 



Lessons Learned 



Although Authorware? is relatively easy to use, it took faculty longer than expected to 
master it. That and a number of contretemps such as having to re-size each screen to match 
the monitors of newly-acquired computers caused faculty to put in approximately 50 hours of 
preparation for every one of the program’s 15 hours. An instructional designer would have 
shortened the process considerably, by assisting faculty with screen layouts, font and color 
selection, and proper use of the software. 

Project Continuation 

The CAI program continues to be offered as a large part of the Introduction to Drug 
Literature course, which now carries two credits. Students spend approximately one hour per 
week on the program and one hour in classroom activities, a portion of which is devoted to 
exercise review and question and answer sessions related to the CAI material. The program is 
now available in a Macintosh and Windows version and can be accessed via the Internet. 

Dissemination and Recognition 

Version 2.0 of “Evaluation of Clinical Drug Studies,” which is copyrighted, has been 
distributed to a number of institutions. It is being used in West Virginia University’s 
baccalaureate and doctor of pharmacy curricula, and has been used for distance instruction at 
Oregon State University, for distance and on-site instruction at Duquesne University, and for 
pharmacist training in literature evaluation techniques. 

The subject of a number of presentations (to the American Association of Medical 
Colleges and the Symposium on Computer Applications in Medical Care among others) and 
publications, the program received one of the first three annual “Innovations in Teaching” 
awards from the American Association of Colleges of Pharmacy. 



Project faculty obtained a grant of $5,000 from the pharmaceutical manufacturer Astra 
Merck to work on a condensed practitioner’s version of the program. An expansion to include 
critical evaluation of pharmacokinetics studies has captured the interest of the Drug 
Information Association, which should yield opportunities to use the program in government 
agencies and the pharmaceutical industry. 

- At West Virginia University , the recent conversion to a-six-year pharmacy curriculum 
will place more emphasis on clinical expertise— and therefore on the need to critically analyze 
dmg studies — than was the case in the past. An additional FIPSE grant to institute 
computerized problem-based learning (PEL) at the school of pharmacy should also help to 
continue to change student attitudes toward the use of computer instruction. 

The latter grant is designed to integrate the educational methods of PEL and concept 
mapping with computer technology, to use the advantages of each. Ten multidisciplinary PEL 
cases— including concept maps for case objectives— have been developed and field tested in 
medicinal chemistry^ pharmaceutics, and clinical sciences. The cases are currently in their 
second year of use and evaluation. 

Available Information 



For additional information, contact: 

Marie A. Abate 
West Virginia University 
School of Pharmacy 
P.O. Eox 9520 

Morgantown, WV 26506-9520 
Telephone: 304-293-1463 
e-mail: abate@hsc.wvu.edu 
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rV. CURRICULUM AND INSTRUCTION 
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COLUMBIA UNIVERSITY COLLEGE OF PHYSICIANS AND SURGEONS 
Electronic Basic Medical Science Core Curriculum 



Purpose 

There is a growing realization that only a fraction of all available biomedical 
knowledge can be mastered by even the most able medical students. Beyond the difficulty of 
grasping so many thousands of facts and principles, medical students also need to learn to 
apply them critically while analyzing medical literature. How should medical schools respond 
to this explosive growth of information? 

Innovative Features 

Most medical schools respond by requiring first-year students to sit through additional 
lecture classes, further inundating them with material and eliminating precious time for reading 
and analysis. Rather than overloading the curriculum, Columbia University’s College of 
Physicians and Surgeons used a new strategy for teaching the basic sciences that deliberately 
reduces the volume of core material and engages students in learning through electronic 
means. The project’s creators restructured the basic medical science curriculum into two levels 
of knowledge— basic and advanced. Level 1 comprises a manageable core of basic materials 
for learning fundamental principles, while Level 2 contains a menu of advanced readings for 
critical analysis. 

The faculty furnish the basic principles for Level 1 through syllabi, while students 
explore advanced materials in Level 2 by reading textbooks and articles and responding to 
medical problems. The curriculum runs electronically over the local network and the Internet, 
together with syllabi, texts, medical photos, cell biology and histology slides, diagrams, 
textbooks, and articles. 
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For example, in the case of the red blood cell, Level 1 provides students with the major relevant 
mechanisms and mathematical formulas. In this basic treatment, the complexity of membrane 
transport is not covered — only the common transport processes regardless of body location. It is the 
function of the Level 2 menu topics to help students explore ion transport to each particular organ, 
because each has its own set of transporting structures and regulating signals. 

Evaluation and Project Impact 

Student advisory groups, faculty groups, and surveys of the entire first-year class were 
used to gauge the appropriate size of the core (1 ,200 pages of syllabus per year) and the ease 
of learning the core material on computers. 

One of the assumptions underlying this curricular restructuring was the belief that defining 
an explicit and manageable core curriculum would increase comprehension and retention of basic 
principles. To evaluate its outcomes, test scores of participating students on the National Board of 
Medical Examiners, Part 1 , were compared to those of non-participating students in previous courses. 

Tests of participating students showed that, despite far fewer class hours and pages of 
syllabus, they consistently scored higher than students in the three years preceding the new 
curriculum. In the surveys, students applauded the curricular revisions and the electronic 
format that allowed them to review course materials and prepare for their labs at home. 

While upholding the goals of Levels 1 and 2 learning, the project managed to reduce the 
number of lecture hours on the core by 40 percent, and the size of the course content (syllabus 
reading) by 400 pages. Because class hours were reduced, it was possible for students to schedule 
additional clinical activities. 

It should be noted that, at the time of the project, the national average of student test 
scores increased. The higher quality of medical school applicants, therefore, may have 
affected the outcomes of the new curriculum. Nevertheless, even if the participating students 
performed at the same level as nonparticipants, they gained a comparable educational 
experience with far fewer class hours, syllabus materials, and, probably, lower costs. 
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Beyond the electronic science curriculum, the college installed e-mail and an electronic 
bulletin board for student-faculty communication, as well as a new 24- hour computer facility 
housing 25 new computer stations within the library. According to project staff, had FIPSE 
not funded the computerized curriculum, the new facility would not have become a reality. 

Lessons Learned 



The project alerted faculty to the need for a specific medical curriculum for developing 
critical thinking among students. However, the expectation that students would take the 
initiative in studying and solving the advanced medical problems intended for this puq^ose in 
Level 2 was only partially realized. Even though extra time and faculty consultation were 
provided for these learning activities, it became apparent that students needed more direction 
and had difficulty learning critical analysis on their own. 

Project staff concluded that students need a well-defined curriculum for Level 2. That 
is, they need vocabulary and concepts in order to teach themselves how to evaluate and use the 
information that they reach during their electronic searching. The Level 2 curriculum is being 
modified accordingly. 

Project Continuation 

The students were so satisfied with the electronic format of the new curriculum that 
they lobbied the dean for additional funds to develop other courses on the network. Because 
students found “words” in the new curriculum less helpful than “pictures,” staff are 
considering adding figures, flow-charts and tables to replace written portions of the syllabus. 
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Dissemination and Recognition 



The faculty response to presentations of the new curriculum at the college has been 
overwhelmingly positive, as has the national response of medical organizations such as the 
American Physiological Society and the Basic Sciences Educational Forum, a group of science 
teachers from 120 U.S. medical schools. 



Available Information 



As part of the college’s home page for the new basic medical science core curriculum, 
a help page with special search engines directs students to course materials and histology slides 
pertinent to 26 microscopy labs. This home page also links to various resources such as a 
glossary of terms, practice questions, and interactive images. The home page address is: 
http://cait.cpmc.columbia.edu: 8000/ 

Additional information about the project can be obtained from: 

Herbert S. Chase 

' Department of Internal Medicine 

College of Physicians and Surgeons 
Columbia University 
630 West 168th Street 
New York, NY 1003 
Telephone: 212>305-1857 
e-mail: hcl5@columbia.edu 
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CLEMSON UNIVERSITY 
Cooperative Chemistry Laboratories 



Purpose 

Ask fornier students of introductory chemistry to describe their experiences in the 
laboratory and their answers are likely to include groans of remembered boredom. Hardly 
ever is the first encounter with Bunsen burners and pH meters recalled as an exciting initiation 
into experimental science. 

The reason for this common attitude is that, traditionally, beginning chemistry 
laboratories have been attempts to illustrate principles rather than true experiments. 

Dependent on memorization and requiring little thought from students, these exercises succeed 
one another weekly throughout the semester, every technique seemingly an end in itself and 
with little relevance to any broader concept or purpose. 

The goal of this project was to convert the general chemistry laboratory into a place 
where the metamorphosis from student to scientist can begin as the learner solves problems, 
evaluates and interprets data, and designs and carries out experiments. 

Innovative Features 



With help from the National Science Foundation as well as FIPSE, the project director 
redesigned the existing series of 12 one-period laboratory exercises into a research-oriented 
program where students work in groups on three laboratory projects during the course of a 
semester. As they carry out the projects, which include identification of an unknown 
compound, thermochemistry, identification of an organic unknown, analysis of a mixture, 
electrochemistry, and analysis of colas by spectroscopy, students learn essential laboratory 
techniques in addition to problem solving, collaboration, and written and oral communication. 
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Students are assigned to groups of four for the entire semester. The groups are as 
heterogeneous as possible for gender, race, and age. Each student plays a given role, e.g., 
leader, recorder, etc., and changes roles with each project. 

The projects are open ended. With the teaching assistant’s help and in cooperation with 
members of their group, students devise and control their own experiments. If they need to 
synthesize a compound, for example, they must decide on the reaction to be performed, 
calculate the amounts of reactants to use, and determine reaction conditions. 

Each group works on a separate but related project. For example, in the role of 
researchers working for the Environmental Protection Agency, students are asked to identify a 
compound that has been found in a landfill, find out its properties and synthesize it. Each 
group is given a different compound, which requires students to develop a different plan of 
action, although the groups use the same experimental techniques. 

Students learn techniques in the context of the experiments they design. 

SuperChemLab, a multimedia Hypercard-based program, contains audio, video and text clips 
illustrating laboratory techniques. If, for example, students need to learn how to perform a 
titration, they can access the appropriate video and text from the “techniques” screen. 
However, if students do not know when to perform a titration, the program leads them to 
instructions on when and how titrations are done by posing a series of questions. 

In order for this hands-on, cooperative pedagogy to work teaching assistants had to 
abandon their instructor role in favor of functioning in a coaching and advising capacity. This 
required a major training effort, involving an initial workshop on cooperative learning- 
covering topics from gender speech patterns to appropriate ways of responding to students— 
and weekly meetings throughout the semester. 
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Evaluation and Project Impact 



One half of the 1,300 students enrolled in beginning chemistry were randomly assigned 
to traditional laboratories and the other half to cooperative laboratories. Their attitudes toward 
chemistry and the laboratory experience were surveyed through a laboratory evaluation form, 
and their final examination grades in the lecture section of the course and their retention rates 
were compared. 

The differences between the two groups’ responses to the laboratory evaluation were 
not large. Nevertheless, students in the cooperative laboratories tended to be more positive 
about their experience than their peers in traditional laboratories. 

Because the same lecture examination was administered to all students, it was possible 
to determine whether a correlation existed between performance in the lecture examination and 
the type of laboratory attended. The cooperative laboratories were not specifically designed to 
reinforce the lecture material, and they covered skills that were not stressed in the lecture 
course. Thus, there was a good possibility that the grades of students in the cooperative 
laboratories would suffer. 

In the first analysis, the type of laboratory seemed to have little effect on lecture 
examination scores. When the scores were analyzed by gender, however, women in the 
cooperative laboratories outscored women in the traditional sections by an average of eight 
points. A two-tailed t-test indicated that this was a significant difference in the scores. A 
smaller rise in the scores of the male students was not statistically significant. 

A similar difference appeared in retention rates. Whereas women in the cooperative 
laboratories dropped the course at a rate of 13 percent, the rate for women in traditional 
laboratories was 22 percent. Men in both kinds of laboratories dropped the course at a rate of 
nine percent. 
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Project Continuation 



The cooperative laboratories have been fully integrated into the curriculum and have 
replaced the traditional sections. 

Recognition 

The project has been presented at a number of conferences and workshops as well as in 
journal articles. An instructor’s guide and laboratory manual have been published by 
McGraw-Hill, and SuperChemLab, a multimedia HyperCard application for laboratory 
techniques and methods, is available from Falcon Software. 



Available Information 



For further information, contact: 

Melanie M. Cooper 

Department of Chemistry 

Clemson University 

Clemson, SC 29634 

Telephone: 803-656-2573 

e-mail: cmelani@Clemson. Clemson. edu 
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CLEMSON UNIVERSITY 
Revitalization of Nonstandard Calculus 



Purpose 

Although the last two decades have seen numerous efforts to reform mainstream 
calculus, until recently nonstandard or “business” calculus— which is increasingly being 
required of students in the social sciences as well as business, economics and management- 
had been left largely untouched. 

Business calculus texts, mostly watered-down versions of traditional calculus, reflected 
this neglect. They offered irrelevant content, failed to incorporate realistic applications, and 
did not require students to use technology. Although firms in business and industry need 
employees who can think logically, write well, use technology proficiently, and solve 
problems as part of a team, the available textbooks in nonstandard calculus only addressed the 
first of these skills. 

Furthenuore, the high level of algebraic manipulation demanded by these texts 
overwhelmed non-technical students, who often suffered from math anxiety, had poor algebra 
skills, and could not see how the course would help their careers. The last mathematics course 
these students would ever taice, traditional business calculus failed to reflect the uses of 
mathematics in an infonnation society, and to provide students with the applications they 
would need in their future work. 

Innovative Features 



The project originated in the wake of a FIPSE grant to introduce graphing calculators 
into undergraduate mathematics courses taken by science and engineering students, when faculty 
realized the parallel need to integrate technology into the mathematics courses for non-science 
majors. Soon after they began to address this problem, it became apparent that a much more 
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fundamental transformation would be necessary to make these courses responsive to students’ 
needs. The new project led to the complete reform of the two-semester business calculus 
sequence. 

Project staff believed that what non-science students need is not mastery of algebraic 
technique, but rather conceptual understanding and the ability to interpret correctly the 
mathematics in real-life situations. Therefore, they focused on the relevance of key concepts 
to the world of change, eliminated topics that stifle the mathematical development of 
non-technical students, and designed courses with less lecturing and more student involvement 
in interpretation, mathematical decision making, and team projects. Seeking to create a 
classroom environment where students would be led to construct their own knowledge and to 
conduct their own inquiries, they encouraged dialogue between faculty and students as well as 
among the students themselves, and they included in the course concepts and topics of 
relevance to non-scientific careers. Because the business calculus sequence is taken by 
students majoring in fields from philosophy to animal science, along with a majority of 
business students, the courses featured in similar proportions business topics and topics related 
to other fields. 

The focus of the revised courses— on rates of change and their interpretation in 
non-technical settings, the derivative as a rate of change, and the integral as the accumulation 
of change— is much narrower than that of the traditional approach. The courses feature the 
interplay between the discrete and the continuous, because many real-life situations involving 
change are discrete processes which can often be represented by continuous mathematical 
models allowing the application of the concepts and methods of calculus. When real-life 
discrete situations are represented by continuous mathematical models, the role of algebra 
becomes to describe, and to enable the student to reason about, the quantities that are 
undergoing change. This departs markedly from algebra’s traditional function as a collection 
of organized manipulations used to obtain a numerical answer to a well-formulated problem. 
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Because most students are likely to work in manufacturing and marketing, the courses 
introduce early on linear, quadratic, cubic, exponential, and logistic models and go on to 
provide modeling experiences using real data. The focus, however, is not on the subtleties of 
modeling, but on using elementary models to obtain functional relationships between variables. 
Students conduct their calculus investigations on the functions represented by the models. 

The new courses require the use of graphing calculators to work with real data and to 
develop insight through graphical and numerical representations of problem situations. Faculty 
use the Texas Instruments 82, but students may use the TI-82, the TI-85 or the 
Hewlett-Packard 48G/GX calculators. 

Evaluation and Project Impact 

The external evaluator examined student questionnaires, projects, journals and tests, 
and interviewed faculty at 13 institutions which had tested the materials. He also compared 
the performance of students at Clemson and at six other institutions (two groups of 273 and 
247, respectively) who were taught according to the new approach. 

Overall, the courses resulted in lower failure rates and improved student attitudes (85 
percent of students believed they were learning as much as or more than students in traditional 
courses), although not in improved retention. 

Because available tests concentrate primarily on procedural knowledge, they are not 
appropriate for assessing the range of student performance elicited by the new courses. 
Nevertheless, a small set of test items appropriate to both traditional and project classes was 
developed. The performance of students being taught by the new approach was deemed, on 
the whole, acceptable. The student projects and presentations, on the other hand, exhibited a 
large range of quality. 
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Lessons Learned 



As part of the project, faculty and teaching assistants were trained to teach the new 
approach by means of summer workshops and weekly seminars. An unanticipated need for 
additional instruction arose as soon as the first pilot courses were carried out— the tutors whom 
students were accustomed to hiring to get them through the course had to be trained in the use 
of graphics calculators and in the reformed methodology. 

Both students and faculty were sometimes hampered in the use of the new materials 
by their preconceptions about calculus: students who had been exposed to innovative teaching 
in prior courses reacted better to the reformed approach than those accustomed to traditional 
lecturing. 

Project Continuation 

Currently all 44 business calculus classes at Clemson-more than 1,500 students per 
semester— are being taught according to the new approach. 

Dissemination and Recognition 

The project has been the topic of over 150 workshops and presentations across the 
nation, as well as ten articles, an Internet discussion group, and a Web site. The textbook, 
published by D.C. Heath and Company in 1995, and the brief and complete first editions, 
published by Houghton Mifflin, have been or are currently being used in over 75 colleges, 
universities and high schools. 




The project won the grand prize in the INPUT (Innovative Programs Using 
Technology) international competition funded by the Annenberg/CPB Project and the National 
Science Foundation. The competition focused on programs that take advantage of modem 
technology— calculators, computers, video, multimedia and telecommunications— to improve 
student learning in mathematics service courses. The project also received a grant to spread 
reform business calculus to ten institutions under FIPSE- s Disseminating Proven Reforms 
competition. 

. Available Information 



Further information may be requested from: 
Iris B. Fetta 

Department of Mathematical Sciences 
Clemson University 
Clemson, SC 28634-1907 
Telephone: 864-656-5212 
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DIABLO VALLEY COLLEGE 

A Team-Taught Interdisciplinary Approach to Physics and Calculus Education 
Purpose 

Even the most gifted and dedicated students in beginning physics and engineering 
courses usually lack the mathematical skills required to obtain the maximum benefit from these 
courses. This is because the design of most physics and calculus sequences makes it difficult 
for students to learn the mathematics they need for physics in a timely fashion. 

For example, Physics I students must learn mechanics without knowing vector and 
multivariable calculus, an experience that the director of this project compares to carving wood 
with a screwdriver rather than a chisel. The next course in the physics sequence uses 
mathematical material— including divergence, curl, Green’s Theorem, Stokes’s Theorem and 
Gauss’s Theorem — which is only taught at the end of the mathematics sequence, if at all. 

To be able to cover the beginning physics syllabi, most instructors teach a bare 
minimum of mathematics, which must compete for space with the physics material. The 
mathematical deficits that students incur in these early semesters worsen as they progress to 
upper level physics courses. Unfortunately, many students never do. 

The project sprang from the collaboration of a mathematician and a physicist who noted 

\ 

excessive failure rates among students in the engineering calculus and physics sequences. 

Even those who passed the courses encountered difficulties when they transferred to a 
four-year institution and many changed majors. 

The project’s creators wanted not only to raise the success rate in the calculus and 
physics sequences without decreasing content or expectations, but to increase the number of 
students, especially underrepresented minorities and women, who transfer to four-year 
institutions and complete their degrees in science and engineering. 
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Innovative Features 



To achieve these goals, faculty integrated the curricular content of beginning calculus 
and physics courses and developed a unified approach to education in these disciplines. The 
curriculum they created, the Special Intensive Program for Scientists and Engineers (SIPSE), 
was purposely designed to be adaptable to a variety of institutions. 

The integrated curriculum involved the re-sequencing of physics and calculus courses so 
that they complemented each other and consisted of two courses, each combining calculus and 
physics, team-taught by a mathematician and a physicist. Thus, SIPSE’s first semester combined 
Calculus I and Introduction to Engineering Physics, a new course designed to augment the physics 
background of beginning students. The second semester combined Calculus III (vector calculus) 
and Engineering Physics I (mechanics). Calculus II was reserved for the third semester. 

SIPSE’s I-ni-n calculus sequence placed vector calculus immediately prior to linear 
algebra, which made the more abstract linear algebra treatment of vectors easier to understand 
than the traditional sequence, where students often take linear algebra before vector calculus. 
The new arrangement also placed the Taylor series just before differential equations, so that 
students would remember the series well enough to use it in differential equations. 

SIPSE also altered the sequence of calculus and physics topics within a given semester, 
so that the mathematics was covered before it was needed. This aided progress in physics 
because the student had the appropriate tools, and progress in mathematics because 
mathematical concepts were put into context right away. Students became familiar 
simultaneously with mathematical and physics notations, terminologies, and approaches. 

The new arrangement allowed sufficient time to cover diveigence, curl, Green’s, Stokes’s and 
Gauss’s Theorems, the statistics needed for physics, and the applications of calculus to physics. The 
integration — as opposed to the simple juxtaposition — of calculus and physics allowed the necessary 
flexibility to cover calculus concqDts before they were required by the physics sections of the course. 

For example, most of the first weeks of SIPSE’s second semester were spent on vector algebra and 
vector calculus, with little coverage of physics material until the students had mastered vectors. 
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Students were required to attend cooperative study groups patterned after those in the 
Professional Development Program designed by Uri Treisman at the University of California, 
Berkeley (see Lessons Learned n, p. 67). The groups met for two two-hour sessions every 
week. They consisted of four to six students and a group leader— either an advanced SIPSE 
graduate or an upper-division student from Berkeley. The study groups met at one of two 
different times (but could have met simultaneously), under the supervision of a faculty 
member. The study group leader provided both individual tutoring and, most often, group 
instruction. The leader also served as a role model, discussing his or her experiences in more 
advanced coursework and in transferring to a four-year institution. Group assignments, which 
covered applications of mathematics to physics or subsets of homework assignments, took less 
than two hours to complete, leaving time for discussions and help sessions. 

After SIPSE, students returned to the traditional curriculum to take Calculus n, which 
covers integration techniques and applications, the Taylor series and differential equations. 
They also took Engineering Physics II (electricity and magnetism), which uses material from 
the end of Calculus II, covered the previous semester. SIPSE’ s flexible fonnat allowed 
students to transfer into it, or out of it and into the regular curriculum, at any point in their 
progress— a necessity given the college’s student population, whose work and family 
responsibilities often disrupt their academic programs. 

Team teaching not only allowed faculty to adjust the ratio of time spent on mathematics 
versus time spent on physics on a daily basis but also made it possible for the physicist to 
augment the mathematician’s presentation with an explanation of the applications of the current 
topic to physics. The mathematician in turn contributed to the physicist’s presentations by 
reviewing pertinent mathematical concepts and discussing differences in notation, terminology 
and style. And both instructors jointly discussed such topics as conservative vector fields or 
centers of mass and moments of inertia with single and multiple integration. In addition, 
students were able to witness discussions between faculty which arose spontaneously in or 
outside of class or in the study groups, and this encouraged them to become actively involved 
with the material. 
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Evaluation and Project Impact 



The external evaluator’s study showed that SIPSE students, although at the outset they 
possessed lower mathematical skills than students in the traditional sequence, had significantly 
higher course success (a final grade of C or above) and retention rates than their peers in the 
control group. Keeping in mind that students frequently withdraw because of poor grades, if 
more traditional students had completed the courses, the differences in success rates between 
the traditional and the SIPSE courses would have been even larger. 

In three of the four courses tested, SIPSE students earned significantly higher grades 
than the controls. In the fourth course, there was no significant difference between the two 
groups. 

Women and minorities were more likely both to persist and to succeed in SIPSE than in 
the traditional program. Over the course of the project, the withdrawal rate differences were 
especially remarkable— a mere six percent for women and minorities enrolled in SIPSE versus 
24 percent for all students in the control group. A comparison of the final grades of women 
and minorities in SIPSE courses versus women and minorities in traditional courses shows that 
the SIPSE group earned significantly higher grades for all courses except beginning physics. 

According to the instructors, SIPSE Physics I students performed better than the 
controls on common final examination questions. The scores of SIPSE students on the 
Wells-Swackhamer Force Concept Inventory and Mechanics Baseline tests compared favorably 
with those of students at Harvard and Arizona State. 

Instructors also observed that SIPSE students seemed stronger than their peers in 
post-SIPSE lower division mathematics and physics courses, more confident in their abilities, 
and more likely to transfer to a top-level four-year institution and to graduate in engineering or 
science. SIPSE students reported significantly more self-perceived positive changes in their 
study habits and in the amount of time they spent studying than their peers in traditional 
courses. 
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The project’s most profound effects were on Physics I, which is not surprising because 
both the creation of “Introduction to Engineering Physics” and the rearrangement of the 
calculus sequence had been intended to prepare students to function well in this course. 

SIPSE’s effects on performance in calculus, though significant, were not as dramatic. This too 
was to be expected, because students benefit more in physics from knowing the necessary 
mathematics than in mathematics from learning its applications to physics. 

In the opinion of project faculty, SIPSE courses — with the exception of the new course, 
Introduction to Engineering Physics— were taught at a more rigorous level than traditional 
courses. And SIPSE students had significantly higher grades in Calculus in despite the fact 
that, unlike students in the control group, they had not taken Calculus n. 

Overall, there were few failures among SIPSE students, and many of these were due to 
family or financial problems. Often, these students returned to SIPSE and later did well. 

Lessons Learned 



For successful team-teaching in SIPSE-style sequences, the physics instructor must 
have a strong background in mathematics. The traditional approach to physics is dictated by 
the students’ inadequate mathematics background. But because SIPSE students do not have 
this deficit, the physics instructor must be prepared to use more advanced mathematical tools 
than those covered in the text. Although it is less important that the mathematician have a 
strong physics background, both faculty members must be flexible and willing to relinquish 
absolute control over the class. 

It appears that combining Calculus HI and Engineering Physics I in the second semester 
is essential for the success of the revised calculus sequence. Faculty allowed some non-SIPSE 
students who were not enrolled in Engineering Physics I to take calculus in the I-HI-n 
sequence. The students did quite poorly. 




.» X 
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Study groups were also crucial in the transition from Calculus I to Calculus III. In 
addition to instruction, they gave students support and a sense of community, something not 
often encountered on two-year college campuses. Frequently the groups endured beyond 
SIPSE into more advanced courses and sometimes even into the four-year institutions to which 
former SIPSE students transferred. 

Because instructors team-teaching in SIPSE spent twice as much time in the classroom 
as they would have in an ordinary course, they received credit for two courses, plus two 
credits for the study group. (For every SIPSE course students received credit for a 
mathematics course, a physics course, and an additional unit for the study group.) Although 
the mathematics department was able to absorb this change in load, the much smaller physics 
department found it difficult to adjust, especially when several physics faculty retired and were 
not replaced. 

Project Continuation 

With the exception of Introduction to Engineering Physics, which became the 
prerequisite for all sections of Physics I at the college, SIPSE, despite its success, was not 
institutionalized. Because SIPSE classes were somewhat smaller than usual, the institution 
deemed the program not cost-effective, and it ended with the expiration of FIPSE funding. 
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Dissemination and Recognition 



Project staff presented SIPSE at many meetings across the United States. In addition, 
the project director organized the Consortium for the Combined Instruction of Mathematics 
and Physics, consisting of Adirondack Community College, N.Y.; Auburn University, Ala.; 
Diablo Valley College, Calif.; Dutchess Community College, N.Y.; North Seattle 
Community College, Wash.; Rose Hulman Institute of Technology, Ind.; 

SUNY/Binghamton, N.Y.; and the University of Puget Sound, Wash. All these institutions, 
some of which have grants from FIPSE and the National Science Foundation, either have in 
place or are working toward a combined calculus/physics program. The consortium’s mission 
is to disseminate findings about these programs through interdisciplinary presentations, 
publications, and Web pages. 

Available Information 

Further information may be obtained from: 

David B. Johnson 
Department of Mathematics 
Diablo Valley College 
Pleasant Hill, CA 94549 
Telephone: 510-689-1230 ext. 489 
e-mail: djohnson@dvc.edu 



99 






UNIVERSITY OF MEDICINE AND DENTISTRY OF NEW JERSEY-NEW JERSEY 

DENTAL SCHOOL 

Curriculum Revision for Community-Oriented Dental Education (CODE) 



Purpose 

Traditional dental education aims to develop technically competent dentists to meet the 
needs of their patients. Acquainting students with underserved communities and their 
problems is not a goal of most dental curricula. Typically, after three years of coursework 
students spend most of their senior year developing clinical skills within the walls of the dental 
school. Thus, they get neither practical nor even theoretical exposure to community dentistry 
before graduation and have little notion of career opportunities in underserved communities. It 
is no wonder that new dentists are ill equipped to respond to the needs of those segments of the 
population who do not have access to dental care. 

In recent years, national reports have called for sweeping changes in dental education, 
recommending that schools encourage experimentation with new and less expensive forms of 
education, promote active learning, emphasize community health, and use community-based 
providers in the training of students. 

Self studies and alumni surveys at the New Jersey Dental School confirmed this advice. 
In response, the school developed the Statewide Network for Community Oral Health Care, 
which eventually grew to 1 1 clinics and 23 faculty. The network’s clinics in turn provided the 
venue for CODE, the school’s new community-based curriculum. 

CODE aims to train dentists who can respond to the needs of the community as well as 
individual patients. It seeks to produce practitioners who are technically competent in all aspects of 
general dentistry and to do so in a more time- and cost-eificient manner than the traditional curriculum. 

Through their involvement in community projects, CODE students learn to respond to 
community needs for affordable health care. They become adept at caring for patients as 
members of a multidisciplinary health team, develop practice management skills, and 
familiarize themselves with dental career opportunities in public service. 
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Innovative Features 



In order to provide the substantial extramural, community-based experience that CODE 
demands, the fourth-year curriculum had to be thoroughly revised. A new course on 
community dentistry and a required community service project were added to the senior-level 
didactic courses, and the clinical component of the curriculum was redesigned. Because 
faculty wanted to involve students at all skill levels, they chose each year’s group of eight to 
12 CODE students from all but the bottom quintile of the senior class. 

Three of the clinics in the Statewide Network for Community Oral Health Care, staffed 
with broadly -trained generalist faculty and supplemented by specialists from the dental school, 
became CODE’S clinical education venues. Each clinic is managed by two faculty members, 
who also ensure a good mix of patients and monitor student progress. Both instructors teach, 
care for patients, and engage in professional development activities. Specialist faculty from 
the dental school make regularly-scheduled visits to the clinics. 

The didactic component of the CODE curriculum consists of traditional courses on 
treatment planning, restorative dentistry, oral surgery, and infection contrql. The new course, 
Dentistry in the Community, explores basic concepts in public health, community dentistry, 
current issues in health and social policy as they apply to dental care and basic skills in the 
collection and interpretation of population-based data and in designing oral health promotion 
programs. 

All students work on an organized community service project, in collaboration with 
service agencies near the clinics. Projects have included, among others, health promotion 
lectures to the elderly or to pregnant teens; oral health assessments for home-bound geriatric 
patients; a curriculum for Head Start children, teachers and parents, and dental services for 
the children; and a resource guide on family violence for dental professionals. 




CODE students complete the required didactic courses and conferences at the dental 
school on Mondays, provide care at the clinics for three and a half days a week, and engage in 
community learning experiences for half a day a week. They meet with the instructor of 
Dentistry in the Community and with the CODE educational coordinator every two weeks to 
discuss their progress. 

Because dental training is traditionally organized around specialized programs, New 
Jersey Dental School faculty wanted to ensure that the generalist faculty at the clinics would be 
able to teach all disciplines. Accordingly, each new instructor participated in 70 half-day 
training sessions with seven departments of the dental school. The sessions were designed to 
instnict faculty in the school’s specific way of performing dental procedures and in how to 
implement the competency- and criteria-based grading systems used by the clinical 
departments. New faculty were also trained in small-group teaching, problem-based learning, 
and the CODE philosophy of education. 

Evaluation and Project Impact 

An external evaluator used faculty and student surveys and interviews, examination 
results, and infomiation from the school’s financial and clinical infonnation systems. The 
evaluator compared matched groups of traditional students to CODE students on their 
perfonnance on school competency tests, mock boards, national boards, regional licensing 
examinations, practice management examinations, and patient care efficiency. A comparison 
of student-faculty and student-staff ratios, student clinical production, patient collections, and 
clinic income and expenses yielded insight into CODE’S cost effectiveness. 

Evaluation results show that, over the course of the project, students manifested statistically 
significant increases in positive feelings toward community service and toward CODE. Faculty 
attitudes toward community-based learning and toward CODE also improved. In debriefing 
interviews, students reported that they felt increased confidence and that CODE made for better 
simulation of private practice, more faculty feedback, and closer faculty-student relations. 



Based on departmental competency examinations and mock boards, all CODE students 
were certified to sit for the North Eastern Regional Boards (NERB), compared to 69 percent 
and 73 percent of their matched cohorts. There were no significant differences in performance 
between subjects and controls on board simulations, on the NERB examination, or on practice 
management mid-terms and finals. 

The evaluator documented a significant difference in the clinical productivity and 
efficiency of CODE students. In the dental school, students are responsible for making and 
confirming patient appointments and financial arrangements, as well as setting up and cleaning 
the operatories. There is minimal chair-side assisting, and students must stand in line to obtain 
supplies from the dispensary. In the CODE clinics, on the other hand, the low faculty-to- 
student ratio and the availability of dental assistants, receptionists, and other forms of support 
enables students to treat about 105 patients a year, versus 41 patients treated by matched 
controls at the dental school. Although the clinical production of CODE students was more 
than 2.5 times greater than that of their traditional counterparts, because fees at the dental 
school are higher, CODE students averaged proportionately lower earnings than the controls. 

Analysis of CODE’S financial impact upon the dental school is not yet complete, but 
preliminary data suggest that it is possible for a program such as CODE to be revenue-neutral 
if the community shares in the costs and if an optimal number of operatories is available with 
an appropriate mix of providers (students, faculty, and staff dentists). 

Lessons Learned 



To overcome faculty resistance, which developed despite meetings with department 
chairs and monthly reports to various committees during the period of project design, staff 
began to implement CODE with a pilot. Ten senior students did a one-month rotation at a 
community clinic, came back to campus full of enthusiasm for the new approach, and were 
instrumental in persuading faculty of the merits of CODE. 





104 



Project staff also formed a faculty advisory committee, consisting of one representative 
from each department, to help to implement CODE and to formulate strategies to eliminate 
barriers. The committee allowed critical faculty to express their concerns and to take the lead 
in making the changes they believed necessary. 

Project Continuation 

CODE has been institutionalized and is being financed by patient revenues and by the 
dental school. 

Dissemination and Recognition 

CODE has been the subject of national meeting presentations, a newsletter, a 
videotape, and a number of journal articles. It received glowing reports during the 
accreditation site visit by the American Dental Association’s Commission on Dental 
Accreditation. The project director received the first Stephen H. Deeper Award for his 
leadership of this project by the national chapter of Omicron Kappa Upsilon, the dental 
education equivalent of Phi Beta Kappa. 

Available Information 

For additional information contact: 



Paul Desjardins 
or 

Robert Saporito 

UMDNJ-New Jersey Dental School 

1 1 0 Bergen Street 

Newark, NJ 07103-2400 

Telephone: 512-320-1600 or 973-972-4633 

e-mail: saporito@umdnj.edu 
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SEVTE GLESKA UNIVERSITY 

Training in Culturally Appropriate Interventions for Native Americans 



Purpose 



General ignorance about Indian culture, the reluctance of non-Indian providers to live 
in the isolated regions where most reservations are found, and the scarcity of Native American 
human services professionals have long made effective delivery of human services to Native 
Americans a rarity. 

The human services faculty at Sinte Gleska University— the first tribally-chartered 
institution in the nation to award the bachelor’s degree— believed that they could train their 
students to provide much-needed high quality, culturally sensitive human services to Indian 
communities. 

Innovative Features 

Sinte Gleska University is located on the Rosebud Sioux reservation in South Dakota, 
and serves the people of the Sicangu band of the Dakota. Aware that Sinte Gleska students 
often lack the resources to attend graduate school, the project’s creators determined to revise 
the human services curriculum not only to provide culturally relevant knowledge but to train 
B.A. students to serve as mid-level professionals. In this respect the program departs from 
common practice in the field, which requires a graduate degree to deliver human services. 

The original curriculum was traditional in style and reflected human services programs 
at non-Indian institutions. The revised curriculum requires each graduate to complete 38 hours 
in Indian studies, including classes in Dakota language, thought and philosophy; cultural and 
comparative healing; tribal laws, treaties and government; and issues of dependency among 
Native Americans. 



107 




112 



Project staff undertook a systematic analysis of all course materials, assessing the 
reading level of each text and reviewing it for culturally inappropriate or offensive concepts. 
For example, many abnormal psychology texts point out that most societies pass through— and 
move beyond— an animistic phase. Staff worked with faculty to help students understand that 
the present-day Lakota belief in spirits does not render the believers either primitive or foolish. 

In the counseling component of the degree, after completing the necessary courses 
students counsel each other, and then, under faculty supervision, the public. Each student 
works with one to three individual clients and one or two families before graduation. 

To ensure that future human services providers have resolved their own personal issues 
and will not attempt to do so through their clients, all students are required to undergo 45 
hours of personal counseling. The counseling is provided by two Native American clinical 
assistants, themselves graduates of the human services program. The aim of this intervention 
is not to identify pathologies, but to allow students to confront constructively the forces that 
shape their lives. With their counselor, students scrutinize their self-concepts, personal values, 
interpersonal styles, attitudes toward intimacy and sexuality, communication styles and skills, 
life and career choices, physical health and well-being, relationship to chemicals, and any 
personality issues and adjustment difficulties that may exist. 

Because so few materials exist to assist in the delivery of services to Native Americans, 
project staff decided that it was essential to norm a variety of assessment instruments on the 
Lakota people. This would prevent the labeling of culture-specific responses as pathological, 
something that often happens when assessment is based on majority-oriented instruments and 
interpretations. 

Project staff modified a structured interview of client history, an intercultural 
self-perception scale, a stress questionnaire, a strategic-approach-to-coping scale, a health 
questionnaire, a social adjustment self-report, and a substance abuse subtle screening inventory 
to render them more reflective of Indian experience. 
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Data for nonning these instruments were gathered from three Dakota reservations by 
human services students and some community residents. Data collectors received training 
which included techniques for obtaining responses from traditional, elderly, or 
Dakota- speaking subjects. Their efforts yielded 898 completed questionnaires, each 34 pages 
long. Staff consider the cooperation in producing this volume of data one of the major 
successes of the project. 

To strengthen knowledge and understanding of the Dakota culture among human 
services and university students, project staff organized a series of workshops in which Dakota 
elders, healers and cultural experts made presentations. Students learned first-hand about 
historical and contemporary cultural practices, ceremonies, legends, myths and tribal history. 
They also helped to prepare for and attended sun dances, powwows, sweat baths, naming 
ceremonies, and memorials. In addition, the Rosebud Sioux Tribe and the university 
conducted public awareness forums on such topics as sovereignty, education, 
self-detenuination, violence, and alcohol and chemical abuse. 

Finally, project staff developed a model to help human services providers relate to and 
alleviate the problems of cultural minorities, specifically Native Americans. The model involves 
learning as much as possible about the cultural group to be served, especially its historical and 
contemporary struggles with the dominant society; becoming familiar with the group’s ways of 
structuring time and interpersonal relationships, learning the appropriate cultural sequences for 
conducting business; and being visibly involved in the betterment of the community. 

Evaluation and Project Impact 

The project was assessed yearly by an external evaluator and, at its conclusion, was 
found to have met all its goals. Employers’ evaluations of human services graduates were 
highly positive, especially regarding cultural awareness and sensitivity. Employees rated 
lower in their ability to work independently and with Little structure, but this may simply 
reflect their status as newcomers to the job. 



Lessons Learned 



Initially, human services students resisted the counseling component of the curriculum. 
The sessions interfered with their schedules, and they felt anxious about confronting personal 
issues. . They also feared that the information ‘disclosed during counseling might be used to 
keep them from graduating. The project director and the clinical assistants reassured, them that 
the counseling would be private and subject to ethics and rules of confidentiality and would not 
affect their academic progress but would improve their effectiveness with clients and their own 
well being. The counseling eventually became so successful that students who had completed 
the 45 hours often requested to continue, and students from other departments asked for 
services. 

In one community, project staff and students encountered resistance to the collection of 
data for the norming of assessment instruments. They were given the access they needed, 
however, after they presented the project at a community meeting and explained how the data 
would be used. The initial reluctance is not surprising given that research or data collection on 
reservations has historically been more oriented toward improving the researcher’s career 
prospects than the lives of the subjects. Project staff believe that the reason the project was 
generally accepted was because they were well known and trusted on the reservation. It is not 
possible to succeed in Native American communities unless one is clearly identified with 
efforts to reduce human suffering and improve opportunities. 

One unanticipated problem arose when students began to analyze the questionnaires. 
Native Americans have difficulty generating hypotheses based on questionnaire scores, 
viewing this as unwarranted criticism or pathologizing. Staff attempted to address this by 
giving human services students a course on client assessment and explaining that assessment 
can reveal positive as well as negative traits. 



Project Continuation 



The revised human services curriculum has been approved and continues to be offered. The 
university has employed the clinical assistants, who were originally funded with grant monies, full time. 

Dissemination and Recognition 

Sinte Gleska’s office of student services has been reorganized to incorporate the project 
model for all students of the university. The business department has created a major in tribal 
business and enterprise modeled on the human services curriculum. Other departments have 
adopted the use of community healers, elders and medicine people in instruction, and a Bush 
Foundation grant has allocated funds for furthering this process. 

Project staff have answered requests for information from a number of institutions. 
Salish Kootenai and Lakota community colleges in particular have consulted on ways of 
making the curriculum more culturally sensitive. 

Staff have made presentations to the American Indian Higher Education Consortium, 
the National Indian Education Association, the University of North Dakota Rural Health Issues 
program, the South Dakota Guidance Counselors’ Association, the University of South Dakota 
psychology department, and the National Organization of Human Services Education. 

Avaiiabie Information 



Further information may be obtained from: 

Rodger Hornby 
Human Services Department 
Sinte Gleska University 
Box 8 

Mission, SD 57555 
Telephone: 605-856-4891 




111 




1 . 1 . 



TUFTS UNIVERSITY 

Developing and Evaluating a Curriculum for Exploratory Learning in Ancient Greek 

Culture (Perseus) 



Perseus, a digital library on Greek culture created in 1993 at Tufts, spans the years 
1200 B.C. to 323 B.C., and includes not only Greek texts but English translations, works of 
art, archaeological artifacts, maps, site plans, and other related materials. The originators 
hoped that Perseus would allow students to seek answers to sophisticated questions in the 
classics from authentic primary sources, while providing them with greater access, flexibility 
and independence in doing so. 

The goal of this project was to study the impact of a second, more refined version of 
Perseus in six institutions as well as within the wider community of users, and to produce a set 
of classroom-tested curricular materials. 

Innovative Features 

Nine faculty members at a variety of institutions were given support to integrate the 
Perseus database on ancient Greek culture into their teaching. Participating colleges and 
universities included Holy Cross College, Rhodes College, University of Michigan, Ball State 
University, Wesleyan University, and Tufts University. 

At these sites, Perseus was used in many different ways. For example, students used 
Perseus for both directed and open-ended assignments, for online exams, as a translation tool, 
as the basis for classics “labs,” and as a library resource or supplement to a course. Perseus 
also proved useful as a lecture tool in classrooms, a companion to museum activities, and as 
part of a team-taught online telecourse. 



Purpose 
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Evaluation and Project Impact 



An evaluator from a discipline other than classics, who was familiar with Perseus, was 
enlisted to assess the ways in which its use changed teaching and learning. He focused the 
evaluation not only on the particular core of materials at Tufts, but on their application in five 
other college settings over three years. The primary variables in his investigation were access, 
freedom, and collaboration. Would students work more efficiently using this tool? Would it 
allow them to broaden the scope of their work and collaborate with others? What difficulties 
does its use present for both teachers and students compared to more conventional tools? 

The evaluation centered on the curriculum’s assignments, activities, and methods, and 
their effects on students, instructors, and classics curricula. It used a combination of site 
visits, yearly evaluation meetings of instructors and staff, observation—both live and 
audio-recorded“in classrooms and laboratories, interviews with students and instructors, 
questionnaires completed by students, written structured reflections completed by instructors, 
student journals, and several content analyses of national and international uses of Perseus. 

The six participating institutions used Perseus in a variety of ways. At Tufts, Perseus had long 
been used outside the scope of Greek language courses and general studies courses for non-majors. Ball 
State University used Perseus to visually investigate Greek architecture, weapons, and art, and to add 
iconography and visual resources in multiple courses. At Holy Cross College, students were able to 
pursue original assignments that, without Perseus, could only have been done by persons with years of 
experience. Instructors there noted that Perseus raised their expectations about the scope and quality of 
student work, and improved students’ performance in examinations. In the University of Michigan’s 
laiige under^duate classics courses, students who were experienced and process-oriented (as opposed to 
their grade-oriented peers) rqx)rted finding the Perseus resource base helpful, while the other group 
became annoyed by the overload of information. At Rhodes College, the Perseus-based course relies on 
primary materials and artifacts (i.e., vase paintings, sculpture, coins and architecture) rather than on 
Greek texts to teach history. Perseus was fully integrated and served as the major source of historical 
information and course assignments. The instmctor mixed traditional, content-based exams with 






114 



non-traditional online administration and feedback scoring. At Wesleyan University, students 
put aside suggested topics relating to the course theme and, by pursuing Perseus historical and 
art materials, discovered unusual topics of greater interest to them and to the instructor. 

The combined data from the participating institutions point to four themes in the impact 
of Perseus: 

1 . Perseus amplifies and augments teaching and learning. Perseus brought much new 
content to courses, not only in texts but in images and maps that were easily integrated. It 
allowed instructors to introduce new courses or activities by relating geographic characteristics 
with textual passages. More importantly, Perseus made possible new kinds of student learning 
(e.g., philological and visual investigations), even by students who knew no Greek. 

2. Perseus requires a substantial physical infrastructure investment. At every site, 
hardware and network problems caused frustration for faculty and students, and laboratory staff had 
to be trained to support faculty and student access. These challenges are no different from those 
usually faced when introducing technology into instruction. Perseus woriced best when used 
through the more familiar World Wide Web rather than the Hypercard-based version. Subsequent 
investigations confirm that access through the Web greatly reduces the local support required. 

3. Perseus demands new conceptual infrastructures for teaching and learning. Substantial 
commitments of time are required from instructors— to create assignments and augment lectures 
with Perseus— and from students— to learn how to access materials and follow leads without being 
overwhelmed by the volumes of information available. Perseus also raises questions of how best to 
use class time and what content should be displaced when Perseus material is added to a course or 
curriculum. Instructors also must learn how to judge and react to electronic assignments. 

4. Perseus is bringing systemic changes to the field of classics. There are several 
indicators that Perseus is beginning to change the way that classics is taught and studied. 

There are now more than 50 courses on the Perseus Web site, representing more than a dozen 
colleges and two dozen instructors, and illustrating the penetration of Perseus into classics 
curricula nationally and internationally. New faculty position announcements in classics now 
list computer experience as a requisite. 

O 
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In addition to the evaluation at the participating institutions, Perseus questionnaires 
were administered in ten courses at eight college sites. Among the many variables in the 
questionnaire, “computer system interface” and “learning effects” produced the most 
suggestive results. Statistically reliable correlations imply that the type of system interface 
used is more strongly related to student learning than are students’ previous computer 
experience, age, or frequency of use. The success of this technological innovation, then, 
depends more on the users’ impression of an attractive and usable system and less on their 
demographic characteristics and experience levels. 

A content analysis of the Perseus discussion list shows that among 271 messages from 
83 individuals in 1995-96, the majority originated in the United States. This was not 
surprising, given that this was the first time anything like Perseus had been put on the Web, 
most of the topics dealt with implementation and logistical issues and the program’s 
availability and format, as well as its implications for instruction and the future of classical 
studies. 

The Perseus Web site took off dramatically in 1995-96. A member of the evaluation 
team initiated a second content analysis of the Perseus list messages. He found, for example, 
that over a several-week period in July and August 1996, the site transmitted 3,034,294 files— 
an average of 8,994 per day— from virtually every country in the world. As of March 1999, 
Perseus was receiving about 125,000 hits per day. This extremely high hit rate (by 
comparison, according to Library of Congress spokeswoman Helen Dairy mple, the Library of 
Congress recorded an average of 60 million hits per month in fiscal year 1998 or an average of 
2 million per day) demonstrates the range and size of the Perseus community and the 
increasing role that electronic databases play in the humanities and instruction. 




116 




120 



Lessons Learned 



Major logistical and time barriers impeded the development and use of Perseus. The 
necessary computing hardware is still not widely accessible. Even when it is, some 
non-majors, highly conscious of time-grade tradeoffs, are reluctant to invest time learning a 
tool specialized to one course. To get around this, instructors at several campuses give 
assignments in which students are able to draw Perseus materials from the Web rather than 
having to learn a new tool. Only among major courses that make moderate to heavy use of 
Perseus does the cost in time to fully learn to use the range of services and materials in Perseus 
seem warranted. 

A related problem is that the new technology requires instructors to make fundamental 
shifts in syllabus and teaching strategies. In addition, junior faculty have little incentive to 
devote considerable time to learning new computer techniques, when publishing their own 
research is more likely to impress tenure committees. 

Continuation and Recognition 

The entire edition of Perseus 1 .0 has sold out, and a new version has been released. 
Popular textbooks now include Perseus companion paths; commercial encyclopedias on the 
Web link directly to Perseus for detailed information; and a spin-off company. Classical 
Technology Systems, provides training and support. 

A 1998 FIPSE grant to Tufts aims to broaden the audience for scholarly documents 
using the World Wide Web. It allows humanists with first-rate ideas to create documents 
intended for the general public. Building on the Perseus database, this project will develop a 
new model to help change the culture of publication in the humanities. 



Available Information 



The many syllabi^ course materials and assignments created during this three-year 

period as well as previous years’ evaluation reports may be found at the Perseus Web site 

under “Teaching with Perseus.” The time investments that instructors make in adopting 

Perseus are also described at the Web site: http://www.perseus.tufts.edu/classes/CS.96s.html. 

A number of articles relating to Perseus are also available. 

Additional information may be obtained from: 

Gregory R. Crane 

Tufts University 

Classics Department 

Eaton Hall 124 

Medford, MA 02155 

E- mail : gcrane@ medusa . emerald . tufts . edu 

Web site: http://www.perseus.tufts.edu/Teaching.html 

Web site: http://www.stoa.org 
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UNIVERSITY OF VIRGINIA 

Core Knowledge Foundation: Alliance for Content-Based Materials and Pedagogy 
Purpose 

Academic problems in secondary and postsecondary education often have their roots in 
the early years of schooling. The University of Virginia’s Core Knowledge Foundation, a 
nonprofit organization founded in 1986, seeks to prevent some of these problems by 
demonstrating that academic preparedness depends upon establishing strong foundations of 
knowledge early. The foundation aims to give students from different regions and economic 
backgrounds a common curricular language, and thus prevent the creation of an educational 
underclass. 

The Alliance for Content-Based Materials and Pedagogy was a collaboration among the 
Core Knowledge Foundation, Trinity University (San Antonio, Texas), Hawthorne Elementary 
School (also in San Antonio), and five other elementary schools. The alliance focused on 
implementing a content- rich curriculum, coUaboratively creating lessons to support it, developing a 
cadre of teachers as trainers, and disseminating its materials and results nationally. 

Innovative Features 



The Core Knowledge Sequence, a 100-page curricular guide, puts into place the ideas 
of E. D. Hirsch, Jr. as published in his works, Cultural Literacy and The Schools We Need 
and Why We Don't Have Them, as well as his series of books on what grade school teachers 
should teach and what children should know. The sequence provides a model of a coherent, 
content-rich core curriculum for the early grades as a planned progression of specific 
knowledge in history, geography, mathematics, science, language arts, and fine arts. The 
sequence was designed by a diverse group of teachers, parents, scientists, and experts on 
America’s multicultural traditions. 
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Core Knowledge builds new knowledge on what students already know. It is presented 
to children early, when they are most receptive to it, adapts to a mobile society where students 
frequently change schools, makes up for academic deficiencies in the early grades so as not to 
impair later learning, and has the same academic expectations of all children. 

Students using the Core Knowledge curriculum typically study subjects such as: 

Ancient Egypt, Greece, and Rome; the Industrial Revolution; limericks, haiku, and poetry; 
Rembrandt, Monet, and Michelangelo; Beethoven and Mozart; the Underground Railroad; 
the Trail of Tears; Brown vs. Board of Education] the Mexican Revolution; photosynthesis; 
medieval African empires; the Bill of Rights; ecosystems; women’s suffrage; and the 
Harlem Renaissance. Specific learning outcomes accompany each section of the curriculum. 

Evaluation 



The evaluation of the Core Knowledge project was entrusted to researchers at Trinity 
University in conjunction with the Office of Research and Evaluation of the San Antonio 
Independent School District. The ambitious evaluation design included standardized 
achievement tests, attitude tests, site visits to model schools, collection of baseline 
performance data in model school sites, creation of school and student data profiles, content 
analyses of student portfolios, and measures of student attendance, suspension and discipline. 

Pre- and posttest comparisons of achievement data for matched samples of students 
were used at model and control schools. As much as possible, the control schools were 
selected to be similar in size, teaching staff, physical plan, and composition of student body. 
Many of the standardized tests employed were already in place at each school and not specially 
targeted to the Core Knowledge program. This use of existing evaluative instruments enabled 
both analysis over time and comparison with similar schools within the same district. 

Education Matters of Cambridge, Mass., was enlisted to conduct extended observations 
and interviews with parents, teachers, students, 'administrators and graduate interns. 



Project Impact 



Obviously, the real effects of such a curricular experiment are cumulative, as children 
build solid foundations of knowledge in a clear sequence, year by year. Nevertheless, 1995 
evaluation data from at least five diverse model Core Knowledge schools in New York, 
Florida, Washington, Indiana and Texas illustrate how these schools appear to be raising the 
academic levels of all students, especially disadvantaged ones, and narrowing the gap between 
academic haves and have-nots. 

Core Knowledge evaluation reports suggest dramatic gains in both quality and equity in 
these schools. For example, at Mohegan School, located in the innermost South Bronx and 
serving primarily Latino and African American students, the average improvement per pupil 
over a school year in the Total Language Arts Battery was more than twice the average 
improvement in the district. A comparable improvement in equity (i.e., children enabled to 
perform up to their potential) is implied by data showing that the lowest quartile of students 
made gains 30 percent greater than the gains made by the lowest quartile in the district as a 
whole. Aside from outperforming neighboring schools, Mohegan’s disciplinary problems went 
down and attendance went up. 

Similar improvement in quality and equity was demonstrated by a three-year 
longitudinal comparison of Three Oaks Elementary, in Fort Myers, Fla., with a control group 
in the same district. Although the control school began the study with higher scores, over a 
three-year period. Three Oaks overtook and surpassed the control school. As measured by the 
Total Battery of the California Test of Basic Skills, the mean at Three Oaks moved from 94 
points below the national mean to a point above it. The standard deviation in students’ scores 
narrowed from 65.77 to 33.64, indicating (as did other measures) that the lowest-performing 
students were being raised to the mean. Disadvantaged students made the greatest relative 
improvement. Pupil suspensions dropped to zero and teacher and student attendance increased 
simultaneously with higher scores on standardized reading tests. 
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In 1994, students at Ridge View Elementary in Kennewick, Wash., showed that, on the 
state mandated tests, they improved their reading, language, math and science scores 
impressively above 1993 scores. Social studies was the only area in which students’ 
performance remained unchanged. 

At Highland Heights Elementary in ruraf Indiana, second through fifth graders, taking 
either the Indiana State Proficiency Test or the California Achievement Test, steadily improved 
their test scores since adoption of the Core Knowledge curriculum in 1992. 

Figures 1 and 2 compare 1994-1995 reading performance in grades 3-5 at Hawthorne 
Elementary with other schools in the San Antonio, Texas, District. Hawthorne students in the 
third grade performed markedly below the district average, but by the fifth grade they 
significantly exceeded the district average on the Texas Assessment of Academic Skills. Their 
successive grade-level increases showed stronger upward trends than those for other San 
Antonio elementary schools, confirming Hirsch’s notion that Core Knowledge achievements 
build on themselves and that early deficiencies can be ameliorated over time. 

The Washington Post reported in November of 1996 that test scores soared the 
preceding year in Calvert County, Md., after the introduction of Core Knowledge, moving the 
county from twelfth in the state to sixth. Calvert County is the first system in the nation to 
adopt Core Knowledge for the entire school district. 

Results reported in 1997 from Core Knowledge schools in Colorado and Washington 
show students performing highly on state-mandated exams assessing abilities in independent 
analysis and expression. At High Peaks Focus School in Boulder, Colo., all fourth-graders 
tested above the proficiency standards in reading on the Colorado Student Assessment, and 78 
percent scored above proficiency in writing. Their scores were the highest in the state for any 
public school that is not a magnet school for the gifted. 
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Ridge View Elementary School in Kennewick, Wash., achieved one of the highest 
percentages of students meeting the standard in all four areas on the Washington Assessment of 
Student Learning— mathematics, reading, writing and listening. The test evaluates subject 
matter knowledge, application of knowledge to real-world situations and analytical skills by 
having students read and apply information to a problem and then explain their reasoning 
process. Seventy percent of Ridge View students are either minorities, eligible for free or 
reduced-price lunch programs, or speak a second language at home. 
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Project Continuation and Recognition 

At last count 350 schools nationwide have adopted or are phasing in Core Knowledge 
programs. These schools are remarkably diverse, including urban schools in the South Bronx, 
rural schools in the Mississippi Delta, suburban schools in Florida and Washington, and 
Catholic schools, Jewish academies, and other private day schools. Core Knowledge has 
received widespread media attention. It has been the topic of feature stories on the major 
television networks, and its staff has been overwhelmed by requests for information. 

The sixth National Core Knowledge Conference was held in Denver, Colo., in March 
1997, featuring acclaimed children’s authors, professional scholars, and visits by conference 
attendees to participating Core Knowledge schools. 
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The results of the Core Knowledge experiment show how powerful teaching and 
learning can be when critical content is aligned with skills instruction in the early grades. As a 
working model of a national curriculum, it projects a common vision for academic success by 
detailing explicit, year-by-year recommendations for specific content in a coherent sequence. 
This ladder of themes and ideas allows students to proceed grade by grade to increasingly 
advanced levels of understanding. It breaks through the low ceiling of expectations of which 
public schools are accused by providing all children with an equitable baseline of knowledge 
and equal access to it. 

Available Information 



Aside from the two books by E. D. Hirsch, Jr., mentioned earlier, one may order the 
Core Knowledge Sequence (revised 1995); A School's Guide to Core Knowledge: Ideas for 
Implementation; Books to Build On: A Grade-by-Grade Resource Guide for Parents and 
Teachers; Books 1-6 in the Core Knowledge Resource Series: What Your First [Second, etc.] 
Grader Needs to Know; and What Your Kindergartner Needs to Know and Lesson Plans (22 
lessons for grades K-5 compiled at the Fourth National Core Knowledge Conference). 
Guidelines for seventh and eighth grades have been reviewed by teachers and field tested in 
schools and will be published soon. 

To order any of the above, call 1-800-238-3233 or send an e-mail to: 
coreknow@coreknowledge.org. To join the Core Knowledge Network, Core-Net, send an 
e-mail to: coreknow@coreknowledge.org. The Core Knowledge Foundation has a Web page 
at: www.coreknowledge.org. It contains, among other things, information articles, lesson 
plans, discussion lists, conference announcements and agendas, job postings, and curricular 
resources. 
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For a recent evaluation of the Core Knowledge experience at one of the model schools, 
see Gail Owen Schubnell, “Hawthorne Elementary School: The Evaluator’s Perspective,” 
Journal of Education for Students at Risk, Volume 1 , Number 1 , 1996, pp. 34-39. For an 
excerpt from E. D. Hirsch’s recent book, The Schools We Need and Why We Don't Have 
Them, see “Reality’s Revenge: Research and Ideology,” in ihQ American Educator, fall 1996 
issue. 

For further information, contact: 

John Holdren 
Research and Publications 
Core Knowledge Foundation 
801 E. High St. 

Charlottesville, VA 22902 

Telephone: 804-977-7550 or 800-238-3233 

e-mail: coreknow@coreknowledge.org 



THE CITY UNIVERSITY OF NEW YORK 

Preparing Doctoral Students for Careers in Urban Colleges and Universities 

Purpose 

Until recently, it was not uncommon for graduate students just one summer removed 
from undergraduate school to face their own first class of freshmen. Many did this with, at 
best, the benefit of a short workshop on teaching just before the beginning of the semester. 
Academia, which insists on more than a quarter of a lifetime’s apprenticeship for research, 
seemed to think that, with regard to teaching, this sink-or-swim approach sufficed. 

That attitude has changed, in large part in response to dissatisfaction from students, 
parents and legislators, and well designed schemes for helping graduate students to teach their 
younger peers now abound. This helps not only the undergraduates, who need the best 
possible instruction in their foundation years, and the graduate students, who need jobs and 
experience, but the universities, who have less money for faculty salaries and must therefore 
depend on adjuncts. For example, in the colleges of the City University of New York 
(CUNY) some 6,000 graduate students teach mainly remedial courses for which they bear total 
responsibility, including syllabus design and choice of text. 

The majority of existing faculty preparation programs focus exclusively on the teaching 
component of academic careers and are oriented toward the needs and characteristics of a 
generic undergraduate population. By contrast, the purpose of this project was not only to 
prepare graduate students to teach but to teach urban undergraduates. Originally, the project 
aimed to provide instruction tailored to teaching in specific disciplines. And it wanted to give 
graduate students a sense not only of what teaching would be like but of what it would be like 
to compete for a faculty position, and then for research dollars and for tenure and promotions 
once the position was secured. 
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Innovative Features 



The project offered three 15- week seminars in consecutive years— in social sciences; 
humanities; and science, engineering and mathematics respectively. 

The seminars covered topics such as testing and grading, discipline, and classroom 
management. They explored the relationship between teaching, service and research. They 
discussed how to get and keep a teaching job. They examined the characteristics of urban 
learners— their demographics, language varieties, cultural diversity, academic preparation, and 
persistence and motivation. The seminars also examined ways to serve their students better. 

In addition to a list of readings, graduate students were assigned at least one, and as 
many as four, structured classroom observations followed by a debriefmg with the observed 
instructor. They developed a curriculum vitae and an introductory course syllabus, which 
were individually reviewed by and discussed with the project director. And they compiled 
information on potential funders for their research areas. 

In the semester following the seminar, students participated in a practicum which 
consisted of monthly sessions with project staff and faculty presenters and mentors. 

Coordinated by the project staff, the curricula of the seminar and the practicum were 
designed to accommodate a variety of presenters. In all, 50 individuals from 16 CUNY 
campuses and 30 departments— including faculty, administrators, and representatives of the 
professional union— participated in some instructional capacity. 

From the beginning, the project attracted much attention from students (there were four 
applicants for every available seat); from faculty, who volunteered to teach and mentor and 
sent bibliographies and advice; and from the 17 CUNY undergraduate campuses, who hired 
all but two seminar participants who wanted to teach. The students who were rejected had 
insufficient English proficiency. 
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Of the total 90 participants, 88 completed the seminar and attended the monthly 
practicums the following semester. Only 41 elected to teach— the others were busy with jobs 
or research assistantships, were preparing for exams, or said they were not ready to teach. 

Even fewer— fifteen— elected to be mentored. Of those who chose not to be mentored, some 
cited time constraints, while others perceived mentoring negatively, a perception that the 
literature indicates is becoming prevalent among students at all levels. 

The principal evaluation instrument was an “Opinion Survey” intended to measure 
changes in knowledge, attitude and confidence regarding college teaching. The survey was 
administered to all applicants before the beginning of the seminar but after acceptance 
decisions had been made and again at the end of the seminar. As anticipated, the posttest 
return rate for participants was much higher than for nonparticipants (100 percent versus 32 
percent). 

Another instrument, entitled “Student Survey— Looking Backward,” was administered 
at the end of the project to students who had participated in the first two seminars (in the social 
sciences and humanities) to assess longer-term effects. The return rate was 89 percent. The 
analogous “Presenter Survey — Looking Backward” produced a return rate of 87 percent. 

In addition, at the end of each session students wrote a one-minute evaluation 
identifying the most valuable aspect of the meeting and suggesting improvements. 

Project Impact 

For all groups of participants combined, the “Opinion Survey” pre- and posttest results 
showed a significant positive change in attitude toward college teaching. The seminars did 
not, however, make a statistically significant difference in knowledge about teaching. 
Regarding the confidence that students felt toward teaching, the scores for the participants on 
approximately half the items on this topic increased to a highly statistically significant degree. 
The scores of the comparison students, on the other hand, did not show appreciable changes 
in any category. 
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On the “Student Survey— Looking Backward,” students rated as the most valuable effects of the 
project the range of faculty perspectives with which they came into contact, the practical tips on teaching 
and careers, the exchange of ideas with other students, the information about urban undei^duates 
(several participants became interested in teaching at institutions similar to CUNY as a result), and the 
“politics” of education. Students also enjoyed the interdisciplinary setting— some said that it was their 
first chance to interact with others outside their department since entering graduate school. Eighty-one 
percent of the respondents thought that the program should be mandatory for all Ph.D. candidates. 

As a result of the seminar and practicum, respondents said that they were preparing more 
carefully for courses, allowing undergraduates to engage in more active learning, using varied 
teaching techniques, maintaining high expectations of students, and assigning more writing. 

Respondents to the “Presenter Survey— Looking Backward” likewise saw much to praise in the 
program. They felt that the opportunity to address the students had helped them to clarify their own 
thinking about teaching-related issues, and that the project was of benefit not only to the students, but to 
the CUNY system as well. In fact, several Ph.D. programs, in response to reactions from students who 
had participated in the project, revised their teaching seminars. Two dq>artments that had provided no 
formal pedagogical instmction to their Ph.D. candidates instituted teaching seminars. 

Lessons Learned 



In the course of readings and discussions in preparation for the seminars, project staff 
came to believe that a separate curriculum for the three disciplinary categories— social 
sciences, humanities, and science, engineering and mathematics— was essential. As they began 
planning the sessions, however, it became clear that only one or two of the seminar's 15 
meetings needed to be devoted to discipline-specific topics such as assessing creativity or 
teaching remedial mathematics. Thus, the original curriculum designed for the social science 
seminar was adopted, with minor modifications, for the two subsequent seminars. 
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It was the consensus of the more than two dozen faculty from diverse disciplines who reviewed 
the curricula as well as of the students and presenters that a generic curriculum is not only adequate for 
training future faculty but helps to break down disciplinary barriers and enhances collegiality. 

Given this turn of events, staff expected that the project might be succeeded upon its 
completion by an institution- wide seminar. Paradoxically, this was not to be. Despite evidence to 
the contrary, faculty, either out of territorial feelings or because they continued to believe that 
pedagogy is discipline-specific, insisted on anchoring teacher training in the departments. 

One of the project’s principal strengths, the large number of presenters — which provided a 
huge range of perspectives while ensuring broad faculty support — turned out to have significant 
disadvantages. Not only did the multitude of presenters inflate the administrative workload, but 
students found the inevitable discontinuities and conflicting opinions disconcerting. The format, in 
which a different speaker addressed each session, also provided less opportunity for in-depth 
discussion than a schedule in which a single speaker covered one topic during several meetings. 

Project Continuation 

As the project entered its second year, CUNY’s Graduate School and University Center 
(GSUC) instituted the Graduate Teaching Fellows, a program whose goals and methods had 
been strongly influenced by the early FIPSE seminars and practicums. 

Project Recognition 

In 1993-94 and 1994-95, the project director obtained funding from CUNY to design and 
conduct a program addressing the special interests and concerns of women Ph.D.s in academia. In 
1992, the graduate school received a grant from the American Association of Colleges and Universities 
and the Council of Graduate Schools, supported by the Pew Charitable Trusts, to pveparQ future faculty. 
Since then the university has granted additional funds to refine ways of integrating training for faculty 
careers into graduate programs. All these endeavors clearly build on the FIPSE project experience. 



Available Information 



Additional information may be obtained from: 



Barbara R. Heller 

Center for Advanced Study in Education 
CUNY Graduate School and University Center 
25 West 43 Street 
New York, NY 10036 
Telephone: 212-642-2910 
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CLEVELAND STATE UNIVERSITY 

Technology-Enhanced Accent Modification for International Teaching Assistants 
Purpose 

The technology-enhanced accent modification program (TEAM) addresses complaints 
from students that their learning and grades suffer because they cannot understand their 
international teaching assistants (IT As). The lack of oral proficiency of many ITAs is 
recognized as a major problem at colleges and universities across the nation, where 
approximately 45,000 ITAs teach many of the 15 million students. 

TEAM’S creators sought to make accent modification instruction more accessible by 
developing reliable multimedia software that operates on low cost personal computers. The 
goal was not to teach ITAs to speak just like native English speakers but to help them to 
become intelligible to their students. 

Innovative Features 



On the assumption that the more ways the learner can sense a feature, the more easily 
he or she will learn and use it, TEAM provides multisensory computer instruction and 
feedback so that ITAs can see as well as hear their speech. TEAM’S multimedia also enables 
learners to retrieve, display, and play model utterances of speech features. This is possible 
because recent advances in computers allow program designers to combine the visual images 
of a feature with audio recordings. 

On the computer monitor the user sees model voiceprints showing the pitch and volume 
of each sound in sample phrases. Below the model, the user sees voiceprints of his or her own 
efforts with the same phrases. In this way, ITAs get visual input for aurally shaping their 
speech patterns to fit the models. The visual comparison helps them to eliminate monotonous 
delivery, choppy speech, inappropriate syllable stress, and the omission of certain sounds. 



A criterion-referenced “Accent Survey” identifies the features of accents that need to 
be modified for ITAs to make themselves more easily understood. The curriculum software 
addresses these features^ and a database contains 3,600 (1,800 male and 1 ,800 female) model 
utterances. An instructional component with online assistance helps ITAs to modify their 
accents’ most distracting features. 

The TEAM strategy differs from other approaches to accent modification in significant 
ways. Whereas most other pedagogies do not focus on general speech comprehensibility, 
TEAM addresses the prosodic (pitch, loudness, timing) features of accent as well as the 
pronunciation of consonants and vowels. 

The curriculum software is designed to be used by pre-professional or graduate students 
who act as accent tutors to the ITAs, thus lowering delivery costs. Project staff train the tutors 
to use the design protocol and to adhere to TEAM instructional tactics. The software is 
designed to teach ITAs to assume responsibility for making and maintaining improvements in 
their accents, and it uses off-the-shelf personal computers that are widely available and 
affordable. 

Evaluation 



The software was field tested on novice computer users, and debugged to run reliably 
and dependably. Once a fail-proof version was created that could withstand 2,000 hours of 
use, testing was expanded to 128 students at Cleveland State, Kent State, and the University of 
Toledo. 

Data were gathered on both the classroom performance of students taught by ITAs and 
the oral proficiency of the ITAs themselves before and at several intervals following 
instruction. TEAM students were compared with three groups of teaching assistants receiving 
other kinds of English instruction, especially the commonly used oral proficiency model. 
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Project Impact 



The field test results were more positive than anticipated. They indicated that students 
enrolled in classes and laboratories with TEAM-taught ITAs as instructors performed better 
and obtained slightly higher grades than students enrolled in classes taught by non-TEAM 
ITAs. 

Further, fewer students of TEAM ITAs switched to other same-time classes or 
laboratories. A majority of students (72 percent) in classes taught by TEAM ITAs reported 
that it took little or no effort to understand their instructors, compared to 49 percent of 
students in classes taught by non-TEAM instructors. Seventy-one percent of students taught 
by TEAM ITAs said that their instructors’ speech did not affect their teaching performance, 92 
percent described their instructors’ speech as comprehensible, and 87 percent said that their 
own classroom performance was not affected by their instructors’ speech. 

TEAM-taught ITAs also performed better than non-TEAM-taught ITAs on other 
measures. On the “Accent Survey,” they received significantly higher posttest scores on 
accent improvement. They took less time to make instructional improvements and, several re- 
tests after instruction, they retained their oral proficiency gains better than non-TEAM ITAs. 
(At these re-tests other ITAs regressed to pre-instruction levels of performance.) The 
frequency of mispronunciations was measured by the Sikorski Oral Proficiency Test. Even 
though the TEAM approach devoted much less time to the pronunciation features of speech 
than to the prosodic features, at retest it still significantly decreased ITAs pronunciation errors. 
However, at subsequent intervals, TEAM ITAs lost most of their pronunciation gains, 
compared to ITAs taught under the oral proficiency approach. 
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Lessons Learned 



It took considerable time and cajoling to persuade institutions to retest ITAs for oral 
proficiency after remedial coursework was completed. The same resistance was encountered 
in getting English as a Second Language departments to use and follow the protocol, and 
several test sites had to be dropped because they would not comply. The project director 
concluded that, in academic settings, political and territorial factors are greater obstacles to 
change than technological ones. 

This project demonstrates how advances in technology have allowed speech-language 
pathologists to work more effectively in accent modification. Extracting and displaying the 
acoustic properties of sound is something that technologies do accurately, quickly and reliably. 
An early assumption, that auditory modality alone was not enough to provide students with 
the clues they need to change pitch, loudness, and timing, proved true. Students can now see 
as well as hear their speech. They can see and hear the pitch line rise or fall. They can also 
see the increased loudness and duration of syllable stress and the word endings that they tend 
to omit. These features help them make their speech easier to understand. 

The accent modification technologies turned out to be much more engaging for ITAs 
than drill and practice approaches. Learning and reading graphic displays of their own 
voiceprints and comparing them to those of the model voice actively focuses learners on their 
tasks and holds their attention. 

In addition, TEAM turned out to be remarkably cost effective. It costs $264 to deliver 
instruction to a student using the TEAM approach, compared to $335 using the oral 
proficiency approach. When long term retention of gains in oral proficiency as well as student 
outcomes are factored in, the TEAM approach is approximately 50 percent less expensive. 
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Program Continuation and Recognition 



In 1995, the project received the National Award for Excellence in Teaching English 
from the English Speaking Union of the United States. The judges rated the TEAM project 
superior in all categories of innovation, measurable effectiveness, and overall value. 

In August 1997, the New York Times education section devoted several pages to new 
English language programs, highlighting the TEAM approach at Cleveland State. 
Consequently, the project director has opened a small company. Accent Services, USA, using 
TEAM software to teach non-native professionals in the community in addition to his 
university students. 

Dissemination 



Complimentary copies of the TEAM software were distributed to more than 350 
colleges and universities with the largest enrollments of international students. Lawrence 
Erlbaum Associates, a major publisher, now markets TEAM at an educational price of $495. 
In addition to the field test sites, the software has been adopted for use at the University of 
Memphis, Michigan State University, the University of Wisconsin, and the University of 
Texas- Austin. Articles based on the TEAM project can be found in at least six 
speech-language journals, as well as in national higher education publications. In addition, 
project staff are participating in a FIPSE dissemination grant involving Johns Hopkins and 
three other universities. 



Available Information 



As noted earlier, the TEAM software is commercially available for higher education 

institutions. The project’s final report to FIPSE is available on disc. 

For further information, contact: 

Arthur H. Schwartz 
Department of Speech and Hearing 
Cleveland State University 
Euclid Avenue at East 24th Street 
Cleveland, OH 441 15 
Telephone: 216-687-6990 
e-mail: A.Schwartz@csuohio.edu 
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CALIFORNIA STATE UNIVERSITY, LOS ANGELES 
Learning English for Academic Purposes (LEAP) 



Purpose 

During the last several decades, almost no campus in the country has been immune to 
the influx of students who are less prepared for college work than faculty would like them to 
be. Faced with the problem of presenting college-level material to students who may have 
difficulty reading the text, many instructors simply give lower grades, sometimes causing the 
institution to lose the very students it recruited. 

Other faculty may react by lowering their expectations and awarding college credit for 
work that belongs, at best, in high school. They are in turn accused of compromising 
standards and ultimately harming students by leaving them unprepared for life and work. 

The discrepancy between the faculty’s expectations and the students’ abilities becomes 
acute at institutions where college- ready freshmen are the exception rather than the norm. 
California State University, Los Angeles, is one of these institutions: 25 percent of its 
students enter as “special admissions.” Seventy percent of freshmen are non-native English 
speakers, and 82 percent place in developmental English. 

Project LEAP was conceived as a partnership between faculty and supplemental 
instruction staff to teach academic language skills and improve performance of high risk 
language-minority students in general education courses. This would be achieved by 
integrating instruction in content and in academic literacy. 

Specifically, the project sought to address the needs of international students who had 
studied English in their countries, recent immigrants who had been exposed to English-as-a- 
Second-Language (ESL) instruction in high school, and English-dominant bilingual students 
with poor academic literacy. 
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Innovative Features 



The project emerged out of two decades of campus experience with study groups 
formed in the Supplemental Instruction (SI) model, with the distinction that whereas traditional 
SI includes students with varied skill levels, at California State University, Los Angeles, the 
groups— led by advanced, specially trained students— include only high-risk students. 

During three years. Project LEAP trained faculty to improve nine existing general 
education courses with special techniques intended to help students to master academic English 
and improve their perfonuance. The courses were supplemented by study groups with the 
same goals. 

The courses selected for language enhancement had a history of presenting conceptual 
and linguistic challenges for language-minority students. Each course revision was carried out 
by a team consisting of a faculty member, a peer study group leader, and a language specialist 
experienced in using course content as a vehicle for language learning. 

The teams focused on techniques for improving course lectures and making the 
textbook accessible. The teams taught students how to assemble information, write essays and 
papers, and prepare for exams. They also encouraged students to become actively involved in 
learning. 

The team revised three courses each year. In the fall, course teams participated in a 
quarter-long seminar on the theory and practice of second language acquisition. In the winter, 
the enhanced courses and study groups were offered. In the spring, the course teams 
documented and evaluated their work, and the annual training manual was compiled. 

During the winter term when the LEAP courses were offered, the teams met weekly, 
and all three teams met together at the beginning and then again at the end of the quarter. The 
study group leader and the language specialist attended all course lectures— a departure from 
traditional SI procedure, in which the SI leader attends lectures but has minimal interaction 
with the course instructor. 
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Evaluation and Project Impact 



Project staff compared the final grades of 128 students in LEAP courses who 
participated in study groups with the grades of their 432 peers in LEAP courses who did not 
attend study groups. Study group participants earned an average course grade of 2.64, as 
opposed to 2.56 for nonparticipants. In six out of the nine LEAP courses, study group 
participants obtained grades equal to or higher than those of nonparticipants, and fewer Ds, Fs, 
and Incompletes. This is especially significant given that 55 percent of the study group 
participants were special admissions students, versus only 38 percent of the nonparticipants. 
Both groups had similar winter to fall retention rates. 

Staff also compared the summary writing skills of students in study groups with those 
of college-ready students in regular courses. Study group participants outperformed more 
experienced college-ready students in a summary writing exercise. 

Despite some grumbling about the length of the study group sessions, students in LEAP 
courses felt that the experience had been helpful, particularly in the areas of reading, writing, 
and note taking. They unanimously said they would recommend the Project LEAP courses to 
their friends. 

Lessons Learned 



In the course of the project, staff found that the approaches to academic English 
introduced for language-minority students were useful for all students who were 
unprepared for college, regardless of language background. Furthermore, the lecture 
enhancements did not impede the learning of college-ready students, but improved their 
classroom experience as well. 

Project staff also discovered that it is important for the quality of both courses and study groups 
to have the members of the team— faculty, students in charge of study groups, and language 
specialists— become true partners. Staff identified another beneficial strategy— that of 
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mixing international students, who typically have good study skills and high literacy but poor 
oral and aural English skills, and early immigrant and native bilingual students, who usually 
communicate well but have poor academic skills. 

Instruction which integrates language and content has powerful benefits for language- 
minority students who lack academic literacy skills. When this responsibility is shared across 
the disciplines, these students can acquire necessary academic skills and build positive 
expectations for future academic study, even while course rigor is maintained. 

Project Continuation 

All the original faculty participants who are still at the university continue to teach the 
enhanced general education courses and have incorporated LEAP principles into their other 
courses. These principles have been adopted by all study groups offered at the institution, a total 
of 15 to 20 groups per quarter. Faculty development workshops continue to disseminate Project 
LEAP to additional campus faculty, with particular emphasis on new faculty and their mentors. 

Dissemination and Recognition 

Project LEAP faculty have presented their experiences at conferences in their own 
disciplines and co-presented with ESL colleagues both at the state and national levels. Several 
have had articles describing their course enhancements accepted for publication in such 
discipline-specific journals as The History Teacher and Advances in Physiology Education. 
Others have applied Project LEAP principles to newly identified areas of need, such as the 
training of graduate student laboratory instructors and the creation of instructional 
enhancements suitable for large lecture classes. More than 60 faculty members across the 
campus who teach lower- and upper-division courses in a variety of disciplines have adopted 
the multi-step writing assignment. And in several departments, experience and interest in 
teaching linguistically and culturally diverse students has become a key hiring criterion. 
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Hundreds of copies of the training manuals and instructional videos have been 
distributed to faculty across the nation. In 1994 Project LEAP received another grant from 
FIPSE to train faculty at California State University campuses and nationwide to integrate 
language and content instruction across the disciplines and continue dissemination efforts. 

Available Information 



Further information may be requested from: 

Ann Snow 
School of Education 

California State University, Los Angeles 
5151 State University Drive 
Los Angeles, CA 90032 
Telephone: 323-343-4373 
e-mail: asnow@calstatela.edu 

or 

Janet Tricamo 
Student Services 
Highline Community College 
Post Office Box 98000 
Des Moines, WA 98198 
Telephone: 206-878-3710 
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THE COLLEGE OF WILLIAM AND MARY 
Freshman/Sophomore Advising and Curricular Coherence 



Purpose 

Studies of the academic advising system at the College of William and Mary concluded 
with the unfortunate statistic that most students met with their advisers only once during their 
first two years. Consequently, students had great latitude in selecting their own courses and 
their choices did not correspond to the faculty’s view of a coherent liberal education. 

With FIPSE support, a new advising system was created to meet the need for greater 
intellectual coherence in undergraduates’ academic programs. Its design drew on the 
experience of successful advising models elsewhere and on findings of the state-mandated 
student-outcome assessment project. 

Innovative Features 



The new advising system is built on faculty volunteers who each have 12 to 15 
freshman advisees. At any given time, about a third of all faculty members are involved in 
freshman-sophomore advising. Advisers receive copious information on their advisees, 
compiled in part from an extensive “General Education Profile” survey conducted by the 
academic advising office in the summer before students enter college. The survey asks 
questions about students’ background and experience, intellectual interests, knowledge, 
strengths and weaknesses in general education, academic plans, oral communication skills, and 
attitudes and values. Three meetings with advisers are now mandatory during the freshman 
year—the registrar’s computers do not allow freshmen to register if they have not first met 
with their advisers in the two weeks preceding the registration period. Advisers have electronic 
access to student academic records and demographic data, and the pool of information 
available to advisers is constantly growing. 



O 

ERIC 



147 



To satisfy the new advising expectations, fk^uJty needed pnpert^^ A vari^ of m^hods were 

used to train advisers, such as large meetings to discuss Internet access tools recently made available by 
the registrar and, the most helpful to faculty, focus sessions devoted to particular advising topics. 

A database of curricular histories was created, comprised of six consecutive graduating 
classes. Findings from this database influenced advising practices in interesting ways. For 
example, analyses of course-taking profiles revealed that science majors took more of their geneiul 
education courses later in their college careers than did humanities or social science majors. As a 
result, upper-division advisers in science concentrations became more involved in discussing 
general education issues with their advisees. Other studies showed that indecisiveness about majors 
and career paths was often tied to other problems, and students who showed extreme indecisiveness 
were referred to the Career Services Office for special attention and counseling, in the hope of 
adding coherence and structure to their academic planning. 

Evaluation and Project Impact 



Peter Ewell, a nationally recognized expert in assessment, served as the project’s 

external evaluator in its final stages. In his report, he wrote: 

Several features of the project’s design are noteworthy and helped account for 
the project’s success. First, it strongly reasserted the original (and often lost) 
mentorship role of advisement by making it a significant adjunct to instruction. 
Rather than discharging a purely informational function in helping students to 
understand curricular requirements and how to act them out, advisers, in the 
best cases, became tutors—actively shaping student understanding of the 
connection between courses and their own path of development. Second, the 
effort was profoundly empirical. The initial student descriptive questionnaire 
was a particular success in building an initial profile of individual student 
strengths, weaknesses, and inclinations. . . while providing opportunities for 
students to reflect actively on what they themselves needed to do. 



Further evidence of the success of the new advising system can be seen in a major shift 
in faculty attitudes toward freshman-sophomore advising. Before the FIPSE project, the 
dean’s office had such trouble finding advisers that the dean levied an advising quota on 
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reluctant departments. Today, the quota has been abandoned because faculty actually seek 
advising assignments. It is not uncommon for faculty planning research leaves to reserve an 
advising slot for the year after their return. The college’s focus on advising, coupled with 
special recognition for outstanding advisers and the creation of small ‘professional 
development accounts’ for faculty who advise freshmen and sophomores, has led to a plentiful 
supply of volunteers. Faculty reviews of the advising system and the associated student 
database have been positive. 

Student satisfaction with the new system and with their individual advisers remains high. On 
anonymous surveys, students reported no interest in changing their advisers. That survey finding is 
confirmed by the fact that the number of students who do request a change in advisers is quite small. 

As expected, the number of student-adviser meetings has risen from one in two years to three in 
the freshman year alone, and advisers report a growing level of continuing contact with their advisees 
even after the students have declared a major and moved on to an adviser in their major discipline. 

Concrete evidence of changes in students’ course- taking patterns due to better advising 
is harder to provide, in part because there has been a major change in the college’s general 
education requirements. However, studies of two graduating classes did show a marked 
decrease in the numbers of special appeals by seniors who found they did not have the proper 
array of courses for graduation. This may be ascribed to increased levels of curricular 
planning resulting from the new advising system. 

The project’s repercussions were not limited to the advising system. The most striking 
evidence of its wider influence is the 1993 revision of the undergraduate curriculum. The 
committee reports and public discussions leading up to the final adoption of the new, more 
structured general education requirement were shaped by the findings of the advising project. An 
often-repeated statistic in the curricular debates was the project’s finding that, in four consecutive 
graduating classes, many students had taken no courses that could be construed as involving 
non- Western histories or cultures, that substantial numbers of students had no science experience 
beyond one course, and that a majority of students had no exposure to the fine and performing arts. 
The new general education curriculum sought to address each of these problems. 
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The advising project also influenced a redesign of the college’s registration system. 
Using technology in advising freshmen in the selection of their courses eventually led to the 
creation of a flexible online system to allow advisers to have quick and complete access to 
their advisees’ records. 

The project’s theme of curricular coherence spread beyond general education to 
advising within students’ majors. The college provided special grants for the creation of 
departmental advising handbooks, which include discussion of how the major fits together and 
stress the intellectual themes that run through and unify courses in the major. Today almost all 
departments have such handbooks, some of them available on departmental home pages. 

Project Continuation 

The new advising system and the new Office of Academic Advising are now fully 
institutionalized, having firm budget support from a special allocation from the Virginia 
legislature. Academic Advising was protected from subsequent rounds of budget cuts and 
received enhanced office space. Automated procedures were put in place to ensure easy 
expansion and use of the curricular history and demographic database. 

In addition to the firm funding base, the project enjoys broad support among faculty 
and administrators. In a display of that support, the college’s recent strategic plan identified 
continuation of the advising system as a high priority for the future and endorsed the use of 
academic advising as a tool to build curricular coherence in students’ academic studies. 
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Dissemination 



The project’s findings were presented at three mid- Atlantic regional meetings of the 
National Academic Advising Association, at four meetings of the Virginia Assessment Group, 
and at one national meeting of the American Psychological Association. In addition, members 
of the project team served as consultants to other universities who were planning changes in 
their advising systems and general education curricula and presented talks on advising and 
curricular renewal at AACU and CCAS meetings. Data analyses from the project were used 
in reports to the State Council for Higher Education in Virginia. 

Available Information 



The Office of Academic Advising produces a “Course Selection Guidebook” for 
freshmen to help them plan for their first college semester. The guidebook provides in-depth 
information on course selection, freshman seminars, and courses in various liberal arts and 
pre-professional majors. 

Information about the project and its outcomes may be obtained from: 

Randolph A. Coleman 
Academic Advising 
College of William and Mary 
Williamsburg, VA 23187-8795 
Telephone: 757-221-2476 
e-mai 1 : racole@facstaff . wm . edu 

or 

David Lutzer 
Mathematics Department 
College of William and Mary 
Williamsburg, VA 23187-8795 
Telephone: 757-221-4006 
e-mail : lutzer@math . wm . edu 




151 




VI. INTERNATIONAL EDUCATION 




153 



UNIVERSITY OF CONNECTICUT 
EUROTECH: An International Program in Engineering 



Purpose 



The introduction into Latin America of a car whose name in Spanish means “it does not 
go” is a well-known stoiy of one U.S. firm’s failure to sell automobiles in international markets. 
But such spectacular failures are relatively insignificant compared to the daily, seemingly minor but 



cumulative effects of the inability of most U.S. professionals and businessmen and -women to 
understand the language and culture of their foreign partners, clients and rivals. 

The University of Connecticut was ideally situated to tackle this problem. More dependent 
on international trade than many states, Connecticut hosts nearly 100 German companies. The 
demand for professionals with cross-cultural and foreign language abilities as well as overseas work 
experience is urgent and concrete. EuroTech was formed to answer this need by providing 
practical study in Gennan, linked with academic training in engineering and industrial experience. 

Innovative Features 

EuroTech’s five-year curriculum confers a B.S. in engineering and a B.A. in German. 
It offers courses for first- and second-year German that include some technical topics, 
one-credit modules conducted in German on engineering topics, summer internships with 
Gennan firms in Connecticut, and a six-month internship in Germany. 

EuroTech, which attracts approximately ten percent of beginning engineering students, 
grew out of well-prepared soil. In the years preceding its inception, the German curriculum had 
been redesigned to accommodate non-traditional majors by including special purpose language and 
interdisciplinary courses. The faculty were familiar with languages-across-the-curriculum, 
content-based instruction, and other innovative pedagogies, and some had experience with scientific 
German. One instructor was certified in oral proficiency testing. 
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In the School of Engineering, some faculty members were fluent in German and willing to 
raise their proficiency to a level that would allow them to lecture in special EuroTech courses. 

There already existed a history of cooperation between the School of Engineering and the exchange 
programs of Baden-Wurttemberg and the Fachhochschule Regensburg. EuroTech took advantage 
of this and of the availability of German-speaking engineers in Connecticut industry. Each year 
two German exchange students in the sciences or engineering are hired, part-time, by the EuroTech 
Program to assist with the courses in technical German and with composing and editing the 
newsletter, and to keep the EuroTech homepage up to date. 

The beginning German instruction for engineering students uses the same text as the regular 
course. EuroTech faculty, however, add vocabulary, discussion topics, realia and exercises 
appropriate to engineering students. The two sections of beginning German, with about 20 students 
each, merge into a single section of intermediate, usually because of attrition in engineering. 

In the second year, students begin taking three one-credit-hour, content-based language 
modules. These courses— Recitation in Applied Mechanics, Introduction to the Sciences, and 
Fields of Technology— are taught by language faculty assisted by a German teaching assistant, 
and structured around lectures by German-speaking engineers from Connecticut industry. (In 
Recitation in Applied Mechanics, a module adjunct to Applied Mechanics, two 
German-speaking professors in the School of Engineering offer the lectures.) To prepare 
students for these technical lectures, the speaker submits a precis of the presentation two weeks 
in advance. The teaching assistants acquaint themselves with the material and develop the 
German-English word lists that students will need to understand the lecture. The assistants 
then explain the material to the language instructor, who prepares simplified study materials 
for the students to review before the lecture. Near the end of each course, students give 
technical presentations on topics such as bridge engineering, the jet engine, disposal of toxic 
wastes, and the principles of air flight. During these presentations, they are evaluated equally 
for technical accuracy and linguistic competence. 
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Thus far two interactive computer modules have been developed for EuroTech students 
using hypertext software. One of these, “Ottomotor,” explains the history, construction and 
functioning of the four-stroke engine. Another, “Hubschrauber, ” presents the history, 
principles of function and technology of the helicopter. Modules are planned for each 
engineering discipline. 

EuroTech majors choose their upper-level language classes from regular course 
offerings for the Gennan studies major. The courses have been designed so that the EuroTech 
students can fulfill the requirements for the language major and some of the general education 
requirements simultaneously. 

In addition to the U.S. and German internships, EuroTech includes study trips to 
Germany for freshmen and sophomores, to motivate students early in their language study by 
introducing them to the German educational system and corporate culture. As a result, some 
students opt for study abroad in addition to the six-month internship. 

EuroTech students are encouraged to undergo oral proficiency testing (before going on 
the Gennan internship, students must test at a level of “advanced high”) and two Goethe 
Institute examinations — the Zertijikat Deutsch for second-year or third-year students, and the 
Mittelstufenprufung for those returning from the internship in Germany. 

Supplemental German language instruction for engineering faculty and staff was 
essential to the success of EuroTech. For two years, four engineering faculty members 
received two hours of weekly classes, and three attended intensive language courses at the 
Goethe Institute in Germany. 

Another important factor in EuroTech's success is the extent to which faculty seek 
industry’s collaboration and involvement. An external advisory board composed of German and 
U.S. company representatives as well as individuals from government agencies meets twice a year, 
and provides Gennan-speaking engineers for lecturing in courses, summer internships in the 
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United States, and contacts for Gennan internships. EuroTech’s budget for publicity, student 
scholarships, and program service activities comes from the contributions of companies 
represented on the board. During the early years of EuroTech the project directors met with 
30 companies in Gennany to develop internship contacts. The relationship with those 
companies is maintained through the newsletter and periodic visits. 

Evaluation and Project Impact 

Although beginning and intemiediate EuroTech Gennan courses are enriched with 
technical content, students use the same texts and take the same written, oral and aural 
examinations as their peers in regular courses. A comparison of performance in course 
examinations as well as in oral proficiency testing and in the Goethe Institute Zertijikat prufung 
showed no statistically significant difference between EuroTech and regular students. 

Compared to German majors, EuroTech students become proficient in the language 
later in their undergraduate careers. Language majors typically have some knowledge of 
Gennan when they enter the university, whereas the typical EuroTech student does not. In 
addition, many German majors have spent some time in Germany when they begin their 
studies, and most go abroad during their third year. EuroTech students, on the other hand, 
usually do not go abroad until their fourth year. Once the EuroTech students return from their 
internships in Germany, however, their command of the language is about equal to that of the 
Gennan majors. In general, students doing well in pre-engineering courses also do well in 
Gennan— something that faculty believe may have less to do with particular scientific or 
linguistic predispositions than with general intelligence, motivation and dedication. 

Students indicate that they enjoy all facets of EuroTech, but they spend a lot less time 
studying German than engineering— only ten percent of their time in the first year and 25-35 
percent in the second year. 
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Attrition appears to correlate with GPAs below 2.0. Among students who indicate in 
the first German course that they intend to pursue EuroTech, attrition is slightly higher than 
among those enrolled in engineering alone. Attrition in engineering is about 50 percent. The 
students who are in EuroTech at the end of the sophomore year and file a plan of study as both 
German and engineering majors, on the other hand, have a much lower attrition rate — about 
ten percent. 

A questionnaire designed to look into the causes of attrition revealed that students often 
find college overwhelming, that classes sometimes do not match students’ expectations, and 
that many students have difficulty working methodically. Some EuroTech students leave 
engineering but complete a degree at the university, while others leave the program but 
continue in engineering. Project faculty anticipate that, as EuroTech ages, successful 
graduates as well as interns returning from Germany will help to keep students interested in the 
program. 

Because it was difficult to convey complete information about EuroTech to all the 
advisors in the School of Engineering, an advisor was appointed and trained in each 
undergraduate engineering program to advise EuroTech students. Some students assumed that 
participation in EuroTech automatically entitled them to summer work and an internship in 
Gemiany. It is now made clear to them that these benefits depend on satisfactory performance 
in both the engineering and the language major. 

An evaluation by a faculty member from another department took these matters into 
account but judged EuroTech generally sound, albeit threatened by projected budget cuts at the 
university. It can be concluded from data accumulated so far that EuroTech succeeds in 
equipping engineering students with the language skills they need to communicate about topics 
in their^field. It is not uncommon for sixth semester German students to discuss bridge 
engineering or the mechanics of the jet engine or ask questions about the automatic turning 
rotary, the rotor head of a helicopter, or the radiation levels in forest floors of post-Chemobyl 
Europe. 
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Lessons Learned 



When EuroTech students in Recitation in Applied Mechanics review the three laws of 
Newtonian physics in German, they are already familiar with the content of the lesson, which 
is also directly relevant to their professional interests. In these situations, the topic of 
discussion absorbs the student’s attention, and the language becomes a natural vehicle for 
expression. It is possible for exchanges in these classes, therefore, to gain a degree of 
authenticity that cannot be equaled by the dialogues about generic topics found in many 
language texts. 

The module’s one-hour format, although convenient in many ways, limits the amount 
of material that can be covered, especially when student presentations, language laboratory 
sessions and discussions vie for class time. Although one hour is sufficient for lectures, 
preparation and revision sessions would benefit from longer periods. Because the EuroTech 
curriculum already demands the maximum allowable number of credit hours, it is not feasible 
to add credits to the modules. Project faculty are considering a flexible arrangement where 
some classes would last longer than others, even though this might mean sacrificing one or two 
class meetings during the course of the semester. 

Recruiting pre-college students who can cope with the academic demands of the 
program now requires a considerable amount of time, although this should decrease as 
EuroTech becomes more widely known and recognized among the high schools. Each issue of 
the newsletter is sent to a broader base of potential students in the high schools, and on-campus 
and cable television publicity in some high schools is also planned. 

Fundraising has proven more time consuming than anticipated. Efforts to raise money 
for servicing the program and for student scholarships have seen slow progress at first, but 
with the first graduates’ successful job placements EuroTech is beginning to receive more 
contributions from employers. 
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Because few high schools in Connecticut teach German, EuroTech was designed assuming 
that students would begin the freshman year knowing no German. This has given EuroTech some 
much needed flexibility. Some students enter the program ready for the courses in advanced 
Gennan, while others are attracted to EuroTech later in their undergraduate progress. Although 
course schedules have been developed in the expectation that students will begin EuroTech in the 
sophomore year, they can begin whenever their interest in international engineering develops. 
Students who enter after the sophomore year are informed of the requirements for completing 
EuroTech and assisted in developing an individual schedule. One student, a chemical engineer, 
began EuroTech in his junior year, knowing no German, and finished his requirements through 
study abroad at a German university, with little additional time required. 

Based as it is on collaboration between liberal arts and engineering and between the 
university and industry, EuroTech requires a good deal of faculty time. It is important that 
institutions contemplating similar programs recognize these demands. The program will not 
survive unless the administration acknowledges the value of curricular innovation, creative 
pedagogy, and intensive outreach work, especially fundraising. Without this kind of recognition, 
and its expression through the merit and promotion system, EuroTech-style projects will fail to 
attract the high-caliber, committed faculty that are essential to their success. 

Another critical consideration for such a program is continuity. Personnel changes, 
whether they be temporary, such as sabbatical leave, or permanent, disrupt progress. To 
maintain continuity, it is important that responsibility for the program be shared by several 
individuals. The minimum is two, but a greater number may be better if the necessary duties 
are coordinated and their successful completion rewarded. 

Project Continuation 

There are now 46 students in various stages of the program. So far EuroTech has five 
graduates— two mechanical engineers, one graduate in computer science and engineering, and 
two chemical engineers. All have been in great demand by prospective employers. 
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Dissemination and Recognition 



EuroTech has received funds from the German Academic Exchange Service (DAAD) 
for the study trips. Yankee Ingenuity Initiative, a supporter of high tech projects to improve 
the skills of local labor, funded the hardware and software for the EuroTech multimedia 
laboratory. During its six years, EuroTech has been funded by FIPSE and by the College of 
Liberal Arts and Sciences and the School of Engineering. Most recently the Connecticut 
Department of Economic and Community Development has agreed to take over the School of 
Engineering portion of the funding. 

Project faculty have written articles and given numerous conference presentations about 
EuroTech, especially at meetings related to languages-across-the-curriculum. The EuroTech 
newsletter and Internet home page serve to further disseminate the project. 

Available Information 



Further information may be obtained from: 

Richard P. Long 

University of Connecticut 

Civil & Environmental Engineering 

261 GlenbrookRd. U-37 

Storrs, CT 06269-2037 

Telephone: 860-486-2074 

e-mail: long@uconnvm.uconn.edu 

or 

Friedemann Weidauer 

University of Connecticut 

Dept, of Modem & Classical Languages 

337 Mansfield Rd. U-57 

Storrs, CT 06269-1057 

Telephone: 860-486-3963 

e-mail: weidauer@uconnvm.uconn.edu 
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UNIVERSITY OF MARYLAND, COLLEGE PARK 
International Negotiation Seminars 



Purpose 

The first two years of college are a risky time for students, especially at large research 
universities. There, many freshmen and sophomores wander aimlessly from one large lecture course to 
another, rarely able to engage intellectually either their peers or their instructors or to understand the 
relationship between curriculum requirements and the process of becoming educated. Bored and 
discouraged, too many drop out before their collegiate experience becomes meaningful. 

The plight of these students motivated the creation of the International Negotiation 
Seminars model. This initiative was launched by the International Communication and 
Negotiation Simulation (ICONS) project, which has been teaching students to negotiate via 
multisite computer-assisted simulations since the early 1980s. On the University of Maryland 
campus, ICONS had previously been relegated to upper-division political science seminars, 
where it had been shown to greatly enhance the undergraduate experience of social science 
students. The seminars project brought this mode of interactive learning to lower-division 
undergraduates, through the creation of a new course, “Introduction to International Negotiation.” 
Key aspects of the model were adapted to courses at other participant universities as well. 

During five semesters, more than 400 University of Maryland students and 800 
students elsewhere in the United States and in Europe took part in the “New Europe 
Simulation.” In Maryland, participating institutions included the University of Maryland- 
College Park, and Frostburg State and Morgan State Universities. The Cooper Union in New 
York, James Madison University in Virginia, and Whittier College in California also took part 
in the simulations. The European partners included the American University of Bulgaria; 
Budapest University of Economic Sciences and Eotvos Lorand University, Hungary; 
Novosibirsk State University, Russia; Jyvaskyla Technical University, Finland; and the 
University of Warsaw, Poland. 
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Innovative Features 



The new negotiation course transformed the passive, large lecture situation into a 
dynamic experience for fewer students. The resulting cross-disciplinary course combined 
lectures with small-group seminars and emphasized collaborative decision-making, 
cross-cultural experience, and introduction to technology via synchronous and asynchronous 
electronic communication with students in other universities in Europe and the United States. 

The weekly discussion section, which usually complements lectures in large 
introductory classes, was used instead to prepare students for the “New Europe Simulation,” 
based on scenarios focusing on ethnic, security, economic, and environmental issues. Each 
section of about 16 students was assigned to represent a European country, and with the help 
of a graduate assistant the students researched the issues, formulated policy, and negotiated 
with teams representing other countries. The cross-disciplinary aspects of the course were 
addressed by faculty guest lecturers and graduate assistants from other departments, such as 
economics and environmental sciences. 

Evaluation and Project Impact 

Project staff collected data from 185 University of Maryland students early and late 
during four semesters. Evaluation instruments included “concept maps” and pre- and 
post-questionnaires. Evaluators used concept maps on the topic of migration in Europe to 
learn about students’ prior knowledge, and subsequent changes in and reorganization of that 
knowledge. The maps showed that early in the semester students concentrated on why people 
leave their home countries, whereas in the cognitive maps from the end of the semester 
attention shifted to more complex questions of reception in the host country and continuing 
relations with the country of origin. Late maps show heightened awareness of the interplay of 
domestic and foreign policy and economic and political issues at global, national, and regional 
levels. 
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Students were also asked to elaborate on questions about economic strength, European 
Union expansion, and environmental standards. The difference in quality of elaboration 
between early and late in the semester reached significance for EU expansion and 
environmental standards but not for economic strength. Late responses tended toward greater 
sophistication and revealed awareness of the interrelatedness of problems and issues. On 
certain factual questions, such as “Is your country a member of the EU?” correct responses 
went from 25.3 percent early in the semester to 88.5 percent late in the semester. Other 
questions were harder to code as right or wrong, but students did show more knowledge of 
their assigned country late in the semester. 

Asked how they felt about various aspects of the course, students generally said that 
their knowledge of economics, politics, and the environment had increased during the 
semester. Regarding students’ reactions to the course as a whole, on a 7-point scale the mean 
for “interested” ranged from 5.5 in the lowest semester to 5.8 in the highest. The mean for 
“challenged” ranged from 5.2 to 5.3. The mean for “identified with country” ranged from 5.1 
to 5.6. 

Two queries— “The course helped me to learn to think creatively” and “The instructor 
encouraged knowledge sharing”— matched questions on the College of Behavioral and Social 
Sciences course survey. The International Negotiation Seminars received ratings of, 
respectively, 4.0 and 4.16 on a 1-5 scale on these items, as opposed to mean ratings of 3.7 and 
3.9 for all courses in the college. A 1996 evaluation of courses meeting the Distributive 
Studies Requirement in Behavioral and Social Sciences included a question about the course’s 
ability to elicit active involvement in learning. On this item. International Negotiation 
Seminars received an average rating of 4.26 on a five-point scale, as compared to a mean of 
3.37 for the other courses surveyed. 

Concerning their work in country teams, students said that they were cognitively 
engaged, tried to make sure everyone participated, and listened to others. There appeared to 
be a significant correlation between high involvement in the course and self-reported learning 
from the policy position paper, online conferences, and e-mail exchanges. 
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Assessment of the innovative features of the project early-on revealed a number of 
weak spots. As with most active learning models, some students were prone to 
marginalization in the classroom. Once participant faculty were alerted to this, they responded 
by developing a syllabus that clearly outlined expectations— including quizzes and writing 
assignments — on a weekly basis. Some students were uncomfortable with the grades they 
received for group research and participation. Faculty addressed this problem by giving 
students a measure of direct responsibility in grading, through fonnal peer review and 
self-evaluation. 

It also became clear that students were viewing the visits by interdisciplinary guest 
lecturers as less important than regular lectures. To counter this, project staff set guest faculty 
to work as mentors with groups of students as they did their research. Despite these efforts, 
however, the instructors estimated that about 20 percent of the students on each team remained 
less than optimally engaged. 

Project Continuation 

Since the grant ended, the Introduction to Negotiation course at the University of 
Maryland has been offered once a year. Given the absence of external funding now, faculty 
must rely on “volunteer” guest lecturers and can no longer hire graduate assistants from 
outside the department to assist with the course. However, the core experience of the course — 
a small-seminar within a large lecture course— continues. Students rotate between the Europe 
simulation and another exercise on the Americas. 
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Dissemination and Recognition 



Subsequent to the International Negotiation Seminars Project, ICONS collaborated in 
the implementation of a FIPSE-funded “Negotiation Modules Project” for community college 
faculty and students in California. In 1997, ICONS received FIPSE funding for an initiative 
that is focused on Historically Black Colleges and Universities and Hispanic-Serving 
Institutions. 

In 1994 the Negotiation Seminars project won first place in the Instructional Category 
of the Maryland Association for Higher Education’s Distinguished Program awards. The 
project has been the subject of a number of articles and presentations, including some by the 
evaluator. 



Available Information 



Further infonnation is available from: 

Jonathan Wilkenfeld 
Brigid Starkey 
Project ICONS 

Department of Government and Politics 
University of Maryland 
College Park, MD 20742 
Telephone: 301-405-4172 
e-mail: icons@bss2.umd.edu 
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UNIVERSITY OF MICHIGAN 
The Michigan Business Assistance Corps 



Purpose 



The Michigan Business Assistance Corps was created to assist emerging democracies in 
Eastern Europe to move toward privatization and to provide masters of business administration 
students with an international experience. The corps arranged summer internships for the students, 
who served as consultants to private and public enterprises in Poland and Russia. 

Innovative Features 

FIPSE support began after a successful pilot which sent a group of nine interns to 
Poland in the summer of 1990. The program expanded and, over the course of the project, 51 
consultants worked with more than 50 companies in Poland and Russia. The interns spent 
three months abroad, generally between the first and second years of study. 

Every fall, the program was promoted among first-year MBA students. This resulted in 85 
to 100 applications, consisting of a resume and a short essay. The project director interviewed 30 
to 40 applicants, and the final selection was made by the host companies in late spring. 

The university’s Center for Russian and Eastern European Studies provided a three-week 
orientation. It included “survival” Polish and an overview of Polish history, culture, politics, 
economics and business practices. Because few interns went to Russia, their training consisted of 
language lessons which students attended three times a week between notification of selection and 
departure and four two-hour lectures on history, culture and contemporary events. 

The host companies and agencies were selected according to the quality of their 
proposed projects, their willingness to offer a meaningful internship, their ability to help with 
housing, and their fit with the applicants’ interests and skills. The project director visited all 
companies prior to the interns’ arrival. 



169 





Interns served in organizations covering areas such as aerospace, arts management, 
agricultural equipment, banking, computers, construction management, public health, 
publishing and real estate development among others. They assisted the companies in 
accounting, bank training, business plan development, cost analysis, economic restructuring, 
health information systems, joint ventures, market research, new product development, quality 
assurance, systems operations and strategic planning. Interns at Procter and Gamble in 
Poland, for example, formulated a strategy for distribution of P&G products to over 60,000 
outlets in that country. Faculty from the Warsaw School of Economics and the Michigan 
Business School provided guidance and support. 

Through donations from corporate sponsors, the Michigan Business School offered 
corps interns a monthly stipend of $1,500 and round-trip transportation. Polish companies 
provided accommodations and $250 a month for living expenses. Russian companies offered 
room and board. 

Before leaving the United States, students contacted their host companies by fax and 
telephone to discuss the preliminary aspects of their assignments. During the internship, students had 
access to a resident assistant at the business school who provided information and data for their work. 

Evaluation and Project Impact 

An external agency conducted the evaluation. In addition to interviewing student 
participants, the agency surveyed nine host companies to assess the impact of the corps on 
each. Evaluators also interviewed faculty and others to judge the effect of the project on the 
business school. 

Most students reacted positively to the internships, with those who went to Poland 
giving the experience higher ratings than those who went to Russia. In general, interns felt 
that they had gained an understanding of and respect for other cultures, a better grasp of the 
difficulties facing the emerging economies of Eastern Europe, and increased self-confidence. 
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Students believed that the internship would help them with future employment, and 
intended to continue the business relationships begun during their stay abroad. They all said 
that they would recommend the program to others, and some stated that the corps had been a 
major reason for applying to the University of Michigan’s Business School. 

Participants thought that the orientation was excellent, but would have liked more 
infonnation on Eastern European business practices. Some also felt that they did not have 
sufficient advance infonnation on the host company and that the company was not prepared for 
their arrival, but they gave the match between the company’s needs and their skills and 
interests high ratings. They rated communication with the company as good, despite language 
difficulties. 

Most students felt that they had made a worthwhile contribution to their company and 
had learned a lot both personally and professionally. They were less positive about the 
management of the corps, and many expressed the need for more administrative support from 
the program while abroad. Almost all felt that the experience had enhanced their ability to 
work with individuals of another culture, their knowledge of international business and their 
consulting skills. 

In turn, most companies were enthusiastic about the interns and wanted to continue 
hosting corps students. After accepting interns for several years, some companies began 
asking for students with specific skills. Several companies offered their interns jobs upon 
graduation. Some companies said that they planned to have their employees attend 
management workshops at the University of Michigan and expressed the desire to house a 
business school representative at their site. 

The interns were thought to have good preparation in business and consulting skills. In 
contrast with the favorable impressions of the training on the part of U.S. faculty and students, 
however, none of the host companies rated the preparation of the interns as excellent, and they 
considered their language skills poor or fair. The consultants’ knowledge of the country’s 
economic system and the local customs was rated as fair. Along with the interns, the host 
companies indicated the advisability of a longer training period. 
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According to the companies, the interns learned quite a bit about the country’s business 
practices, economic system and customs during their stay, but made less progress on language. 
Company representatives felt strongly that, given the magnitude of the adjustment the students 
had to make, the internship was too short. 

Since several managers of the Polish host companies were subsequently appointed or 
elected to senior positions in the government, the corps had some effect on public policy. 

After one such manager became prime minister, he invited all interns to his office to 
acknowledge and honor their work. 

The internationalization of the University of Michigan Business School’s culture 
constituted the major contribution of the corps, according to faculty. The corps heightened the 
faculty’s awareness of the globalization of business and the proactive posture of U.S. business 
toward emerging economies. Business cases prepared by returning interns were used 
informally by MBA faculty, who also called on interns for their practical knowledge of 
business practices in Eastern European economies. The corps also made contacts between 
faculty and business firms and government agencies in Poland. 

Business school faculty believed that, as projected by the creators of the program, the 
corps benefited both students and companies. While the interns helped the companies to apply 
the latest business concepts and practices, the Eastern European business leaders provided 
lessons in grass-roots entrepreneurship. 

Finally, the corps helped the business school to gamer support for the Davidson 
Institute— which features shorter internships with large firms and considerable faculty 
involvement by demonstrating that interns could provide assistance to businesses in 
transitional economies. A number of faculty felt that the corps would lose its fundamental 
character if it were absorbed by the Davidson Institute. 
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Project Continuation 



The corps was merged into the Davidson Institute, and students presently have a choice 
of two types of internships. 

Discussions have begun about incorporating the corps experience into the business 
school’s Office for the Study of Private Equity Finance, which conducts research, teaching and 
service in financial innovation and entrepreneurial activity. 

Dissemination and Recognition 

The success of the MBA Corps gave rise to the Domestic MBA Corps, whose function 
is to assist community nonprofits to perform their mission. The Domestic MBA Corps placed 
23 interns in community agencies during its first two summers and plays a leading role in the 
implementation of an Americorps program. 

A number of institutions, including the College of William and Mary, the University of 
Maryland, and Ohio State University are modeling international internship programs on the 
corps. 

Available Information 



Further information is available from: 

Marian J. Krzyzowski 
The University of Michigan 
School of Business Administration 
Ann Arbor, MI 48104 
Telephone: 313-998-6201 



SAN DIEGO STATE UNIVERSITY 
The MEXUS Program 



Purpose 

The internationalization of business education has for the most part proceeded at a 
cautious and conservative pace. Originally, schools responded to the expansion of 
international business by offering courses in international marketing, finance, and 
management, all taught from a thoroughly U.S. perspective. Business students rarely traveled 
abroad. Eventually international study and internship opportunities began to appear in the 
business curriculum, but usually they were short and did not give students the in-depth training 
and experience they needed to do business in another culture. 

Along with other FIPSE-funded projects, MEXUS is in the vanguard of innovative 
international study programs. As the first undergraduate transnational dual degree program in 
the United States and Mexico, MEXUS provides total immersion training with the aim of 
producing not Just American businessmen and -women with knowledge of international 
business but tndy international managers. 

Innovative Features 



MEXUS is a binational undergraduate international business curriculum which 
integrates business, foreign languages, and regional and cultural studies. Through a consortial 
arrangement with universities on both sides of the border, it allows Mexican and U.S. students 
to satisfy degree requirements for the B.A. in international business as well as the licenciatura 
en negocios internacionales simultaneously. Students from both countries live and study in 
Mexico and in the United States a minimum of two years each, and complete an internship at 
the end of the second year in the host country. 
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The U.S. MEXUS partners are San Diego State University (SDSU) in San Diego and 
Southwestern College (SWC) in Chula Vista, Calif. The Mexican partners, both located in 
Tijuana, are the Centro de Ensenanza Tecnica y Superior (CETYS) and the Universidad 
Autdnoma de Baja California (UABC). 

The four partner institutions, located within a 15 mile radius, are members of large 
academic networks— SDSU is part of the 21-campus California State University system, and 
SWC is one of the 100 campuses of the California Community College system. Because 
academic requirements are articulated across campuses and course credits automatically 
transfer, MEXUS is easily accessible to vast numbers of U.S. students. On the Mexican side, 
CETYS is one of the founding institutions of the Federacion de Instituciones Mexicanas de 
Educacion Superior, a national network of prestigious private universities, and UABC is part 
of the university system of Baja California. 

The creators of MEXUS had completed a great deal of groundwork before approaching 
FIPSE, gamering administrative support and drawing up preliminary agreements among the 
partners. After the FIPSE award, they finalized the consortium agreement, established the 
curriculum, recruited and selected students, and hired a coordinator, located at SDSU, to work 
with the four partner institutions and administer the project. 

U.S. students who participate in MEXUS are overwhelmingly Hispanic— 73 percent are 
Latino, 15 percent Anglo-Latino, and 12 percent Anglo. They demonstrate above-average 
academic performance, both in high school and college. Their average high school GPA is 
3.6, and their class standing averages at the top eight percent of the class. The SAT average 
of MEXUS students is 120 points above the SDSU average. Their average college GPA is 
3.4, versus an average SDSU GPA of 2.64. 
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Evaluation and Project Impact 



During the project, MEXUS reached its enrollment goal of 120 students. 

Project staff intended for all U.S. MEXUS students to take the modified CETSCALE 
Questionnaire, the Visual-Oral Communication Instrument, and the American Council of 
Teachers of Foreign Languages Oral Proficiency Interview before and after their stay in 
Mexico. It has, however, proved difficult to persuade returning students to take the posttests. 
In the case of the four students who have graduated to date, neither the cultural values nor the 
language proficiency posttests show significant change. This may simply be a reflection of the 
students’ Hispanic background. 

Lessons Learned 



Given their experience with the project, the creators of MEXUS feel strongly that it is 
best to begin this kind of effort with a single partner. Multiple partners can complicate a 
program exponentially: in three years of FIPSE funding, MEXUS staff had to develop 1 1 
different curricula. Every time a partner makes a change in the curriculum, the programs for 
new students change, and all students must be advised individually to ensure that they will be 
able to fulfill graduation requirements in a timely manner in both countries. This can prove 
overwhelming for the staff, and ultimately compromise the quality of instruction. In 
closely-integrated programs such as MEXUS, more is certainly not better. 

Coordinating systems across such diverse institutions— public, private, two-year and 
four-year— proved more challenging than expected. This difficulty was compounded by 
differences in national, socioeconomic and academic cultures, in federal and state requirements 
in each country, in administrative procedures and in institutional missions. Each partner had 
to change many established policies to accommodate the project. The principal problems 
encountered at each institution exemplify the kind of stumbling blocks that projects such as 
MEXUS are likely to encounter. 
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At SWC financial aid is tied to enrollment in classes, and since the college has never 
had an international exchange program before MEXUS, there is no mechanism to document 
registration on the home campus while attending classes abroad, rendering MEXUS students 
ineligible for financial aid while they study in Mexico. To solve this problem, SWC students 
transfer to SDSU before beginning their Mexican studies and receive financial assistance from 
SDSU. This of course requires students to plan unusually far in advance. 

In addition, while the consortium policy is that partners waive tuition for exchange 
students— students pay only at the home institution— this is not possible at SWC. SWC’s board 
of regents approved in-state tuition for Mexican students, but this still meant that they had to 
pay two tuitions. In 1997 CETYS and UABC decided to eliminate home tuition payments for 
their students attending SWC. 

At SDSU, when MEXUS began students studying abroad, they were eligible for 
financial aid for a maximum of one year only. The vice president for academic affairs 
successfully petitioned the Western Association of Schools and Colleges to allow SDSU to 
offer financial aid to MEXUS students during their second year abroad. 

There was some disagreement among SDSU faculty about the appropriateness of 
granting two degrees— the B. A. and the licenciatum— for a single body of coursework. Using 
the European Union's SOCRATES and ERASMUS as points of reference, MEXUS staff 
justified the dual degree by arguing that: 1) The two degrees represent the same level of 
achievement, and thus should be considered “duplicate” rather than “additional.” 2) The 
language skill demanded of MEXUS students is greater than that required for a regular degree. 
3) MEXUS duplicate degrees require more credit than either the B. A. or the licenciatura 
alone. 4) Students could in theory complete the two degrees independently and without the 
knowledge of either institution. 

UABC faculty were developing a degree in international business at the same time as 
they were instituting MEXUS. This taxed their academic and administrative resources and 
caused some of the classes that MEXUS students needed to be unavailable. These problems 
have since been resolved. 
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On the U.S. side, MEXUS has had difficulty recruiting students of non-Hispanic 
background, while on the Mexican side families are reluctant to let their daughters reside in 
the United States. Students from both countries have struggled to adjust to drastically different 
approaches to higher education— in Mexico, the European-style habit of placing responsibility 
for learning squarely on the student’s shoulders, versus the structured, heavily monitored 
approach adopted by U.S. faculty. 

Project Continuation 

MEXUS has been institutionalized at all four partner institutions, and the coordinator’s 
position at SDSU has been made full-time and permanent. The coordinator has been charged 
with expanding the dual degree to other disciplines and other countries. SDSU is currently 
finalizing curriculum and program design with institutions in France, Quebec, Spain and 
Gennany. Grant applications have been submitted to develop dual-degree international 
business programs in Brazil, Chile, China, Japan and Russia. 

In fall 1997 the MEXUS/Women’s Studies program, involving SDSU and UABC, was 
inaugurated. Graduates will obtain a B.A. in Women’s Studies and a licenciatura in either 
sociologi'a, economfa, or psicologia. 

Project Recognition 

MEXUS has been cited by Presidents Clinton and Zedillo and by the U.S. State 
Department as an example of United States-Mexico cooperation. 
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Available Information 



Additional information is available from: 

Michael Hergert 
MEXUS Program 

San Diego State University, BAM-427 
6500 Campanile Drive 
San Diego, CA 92182-7734 
Telephone: 619-594-7297 
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WRIGHT STATE UNIVERSITY 
Project Chile 



Purpose 

With one third of U.S. corporations engaging in international business, and the state of 
Ohio a leader in the export of manufactured goods, it was clear to the faculty at Wright State’s 
College of Business and Administration that the preparation of their students had to change to 
reflect these circumstances. 

Based on intensive preparation leading to an international internship, Project Chile was 
intended to give students technical import-export skills, fluency in Spanish, an acquaintance 
with Chilean culture, and professional work experience. The project’s creators also wanted to 
benefit participating companies, in the hope of eventually increasing the economic development of 
Chile’s Lake District or ninth region and of Miami Valley, Ohio, where WSU is located. 

Innovative Features 



Project Chile was developed in collaboration with the Corporacion de Desarrollo y 
Energia— a Chilean organization that served as a link to Chile’s business community— and the 
University Studies Abroad Consortium of which Wright State University is a member and 
which has a site at the Universidad Andres Bello in Santiago. 

After being accepted into the program, each student was assigned a Chilean company to 
research as part of two preparatory courses at Wright State— “International Trade Management” 
and “Chilean Culture and Commercial Spanish.” Students then went to Chile, where they 
spent the first four months in a language immersion program at the Universidad Andres Bello. 
The program culminated in a six-month international trade internship with the Chilean 
company that each student had researched. 
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Interns performed such tasks as analyzing the effects of Mercosur (a common market 
agreement among Argentina, Brazil, Paraguay, and Uruguay and, by the year 2000, Chile) on 
various Chilean industries; developing import-export manuals; participating in negotiations to 
establish joint ventures between Chilean and U. S., Canadian and Spanish fimis; negotiating 
and processing letters of credit for international shipments; and analyzing the potential of 
overseas markets. 

Evaluation and Project Impact 

Although more than 100 students expressed interest in Project Chile each year, and 67 
took the International Trade Management course and participated in domestic internships 
focusing on international trade, only eleven— three of them from other institutions— undertook 
the full project experience. 

Language competency gains were measured by written and oral pre- and posttests. The 
pretest scores were higher in the written than the oral examinations, an outcome to be expected 
given the emphasis on written communication in college foreign language courses. After the 
Chilean Culture and Commercial Spanish course, students scored at the “Intennediate-mid” 
and “Intermediate-high” levels on the American Council on the Teaching of Foreign 
Languages (ACTFL) business writing examination. Upon returning from Chile, all but one 
scored at the “Advanced” or higher levels. Students also made significant gains in oral 
communication— two to four levels on the ACTFL scale, as measured by the Video Oral 
Communications Instrument. A full set of scores was not available for every student. 



An intercultural adaptation instrument was especially designed for the project, to be 
used as a pre~ and posttest. The instrument was filled out for every student by the director of 
the University Studies Abroad Consortium program and by the internship supervisor at each 
company. Although students achieved high intercultural scores in the pretest, indicating that 
they had been well prepared at their home campus, they made further gains both after the 
language immersion and the internship periods. 

Host companies reported that the work of the interns exceeded their expectations and 
proved their appreciation by accepting subsequent interns. 

Lessons Learned 



Project Chile’s paramount difficulty was in attracting students. Staff surmise that the low 
participation was principally due to the program’s length— two years from application to 
completion, with one full academic year spent in Chile. And because the business school had no 
foreign language requirement, most students lacked the necessary language skills for the project, 
and had to add a full academic year and a summer of language study to their preparation time. 

Such a schedule clearly requires careful planning by students from the beginning of their 
undergraduate careers, at a time when they may not be certain of even their major, much less that 
they want to study abroad. It is likely that the newly created international business major, which 
requires over two years of foreign language, will lead to increased participation in Project Chile. 

Because so few students went abroad, project staff offered to those who remained on campus the 
opportunity to participate in J. A. Globe, an international trade simulation. Conducted in collaboration 
with a Chilean university, the simulation attracted 24 U.S. and 40 Chilean students. Ten Wright State 
students also participated in domestic international trade internships with local companies. 

There were also difficulties in collecting evaluation information. Host companies were 
extremely reluctant to submit written evaluations of students, fearing that students would have 
access to them and that negative evaluations would reflect on the companies. After staff 
explained the confidentiality of the process, however, the companies fully complied. 
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Several of the students who went to Chile failed to complete the posttests. Students are 
now required to sign a contract committing themselves to completing all tests or face expulsion 
from the program. 

One of the project’s original goals was to demonstrate that participants would be hired 
in positions requiring international trade experience and language skills. However, because 
most students elected to go on to graduate school, this goal could not be achieved by the end 
of the project. In the case of five of the students, nevertheless. Project Chile opened the doors 
to various opportunities related to the study of international business. 

Project Continuation, Dissemination and Recognition 

At the end of the FIPSE period, a grant from the National Security Education Program 
provided an additional two years of funding to enhance Project Chile. A modified version of the 
project will continue in collaboration with the University Studies Abroad Consortium and the 
Universidad de Concepcion, Chile. The consortium offers increased publicity and a larger pool of 
participants, as well as the possibility of expansion to other countries. Although Wright State 
University will continue funding stalf salaries for Project Chile, after the National Security 
Education Program ends programmatic moneys will have to be raised from external sources. 

The project was recognized with several group and individual awards for its results. 

Avaiiabie Information 



Additional information may be obtained from: 

Robert F. Scherer 

Division of Community Programs 

College of Business and Administration 

3640 Colonel Glenn Highway 

Dayton, OH 454324-0001 

Telephone: 937-775-3718 or 937-775-2814 

e-mail: robert.scherer@wright.edu 
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VU. TEACHER EDUCATION 
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UNIVERSITY OF CALIFORNIA, RIVERSIDE 
Learning at Many Levels: Teacher Preparation at Professional Development Schools 

Purpose 

In traditional teacher education programs, prospective teachers study their academic specialization 
in a university department, are trained in pedagogy in the school of education, and devote a semester or a 
quarter of their final year to student teaching. Once they begin their first teaching job, opportunities for 
professional development or research outside their school’s in-service prograjns are limited. 

This project sought to create professional development schools (PDS) that would 
prepare future teachers in the most realistic context possible and would integrate the subject- 
matter, pedagogical, and experiential aspects of their training. And because being a teacher 
should be synonymous with learning, the project’s creators wanted to provide professional 
development opportunities for experienced teachers and to encourage research related to 
educational practice on the part of teachers and university faculty. 

The project was carried out by the University of California, Riverside’s Comprehensive 
Teacher Education Institute (CTEI), a collaborative partnership of the university’s School of 
Education, selected academic departments and local schools. 

Innovative Features 



The project began as a pilot at a single high school. During this phase, CTEI staff 
developed strong collaborative relationships with the school’s teachers and administrators, 
designed specific features of the project, experimented with various approaches, built a 
reputable program, and attracted interest from other schools. 

Three more professional development schools were added during the years of FIPSE 
funding, with the assistance of an 1 1 -member team of university and school faculty and 
administrators. The team reviewed applications from schools for evidence of receptivity to 
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change and collaboration, professional cohesiveness among staff, interest in research, and 
commitment to teacher-driven staff development. They also evaluated applications for 
cross-cultural diversity, community support, and involvement in restructuring. This last item, 
they reasoned, would allow project personnel to complement the school’s efforts, make 
changes quickly, and increase the likelihood of institutionalization. 

The preparation of prospective teachers at professional development schools is a 
year-long process, which students undertake after completing the B. A. degree. During their 
fifth year, student teachers enroll full-time as graduate students at the university and pay 
regular tuition. They receive no compensation for their work at the PDS, but are eligible for 
financial aid and substitute teaching. The program includes 21 quarter units of field 
experience in the schools and 26 quarter units of courses. Some of the courses are taught at 
the university and some at the PDS sites, which are also wired for distance learning. 

The program by necessity follows the school rather than the university calendar. In 
order to foster mutual support and solidarity, six to twelve student teachers are placed at each 
school. They attend teacher work days before classes begin, and their time at the school 
increases gradually to full-time in the spring. Their early induction and involvement 
throughout the year helps them to be accepted as faculty by both teachers and students and 
gives them a realistic perspective on the demands of teaching. 

A university resident supervisor is assigned to each cohort of student teachers at a school. 
This individual, who is especially hired and evaluated by the university, may be a teacher, a faculty 
member, or a doctoral student. He or she works primarily at the school rather than on campus and 
is based in a room permanently assigned to professional development purposes. Like the student 
teachers, the resident supervisor follows the school rather than the university calendar, works with 
cooperating teachers, and has frequent contact with administrators and students. 

Cooperating teachers apply to work with prospective teachers. Those selected are 
trained by the university supervisors and other cooperating teachers at the site. To foster 
consensus and commitment, the cooperating teachers and the university supervisors jointly 
develop guidelines about the teachers’ specific roles and responsibilities. 
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Aided by a handbook and daily seminars, student teachers engage in intensive guided 
observation in a variety of classrooms and subjects for the first three weeks of the year. Then 
they are assigned their cooperating teachers. Although the final assignments are made by the 
university supervisor, both cooperating teachers and prospective teachers have a say in the 
decision. The latter typically work with more than one cooperating teacher. 

After the initial period of obseivation, student teachers focus their attention on the classes for 
which they will eventually take full responsibility. They begin by assisting students in small groups and 
progress to grading woii< and teaching weekly lessons. In January they teach a week-long unit, and 
when the school’s second semester begins in February they assume full responsibility for their assigned 
classes. This gradual transition results in an easier, less anxious experience for everyone involved. 

Daily seminars— Jointly planned and led by the cooperating teachers and the university 
supervisor— are held at each site. These seminars, some of which are interdisciplinary, vary in 
method and content. They break down the isolation that prevails among secondary school 
faculty and develop cooperation, collegiality, and a sense of community. 

In addition, student teachers meet individually with cooperating teachers and with the 
university supervisor, thus gaining several mentors. Cooperating teachers meet together to 
work on issues such as assessing student teachers. 

Students also take subject-specific instructional methods courses. These courses— some 
of which are offered at the schools— are collaboratively developed and taught by School of 
Education faculty, departmental faculty, and school teachers. 

Unlike their peers in traditional programs, CTEI student teachers are intensely involved 
in self assessment. They keep reflective journals, view and evaluate tapes of themselves, 
develop and distribute student surveys, and compile a teaching portfolio. 

During the school year, student teachers participate in all in-service programs, and they 
attend conferences and regional workshops with cooperating teachers. They participate in 
interdisciplinary team teaching, and they become certified as substitute teachers. When 
cooperating teachers and the university supervisor determine that they are ready, they have 
priority at their school for substitute teaching assignments, which they have the option to 



accept or decline. This arrangement allows prospective teachers to earn money, gain 
experience, and review their performance with the teacher for whom they substituted. The 
arrangement in turn benefits the school by making available substitutes who are familiar with 
its norms and instructional programs. 

To enhance the professional development of experienced teachers, the project focuses 
on activities not typically available to them and strives to complement the schools’ staff 
development programs. Participation in all activities is voluntary and open to any teachers at 
the partner schools. 

Teachers are included in the project management team, along with school 
administrators and university personnel. Membership in the team, which meets bimonthly for 
a full day to develop and refine project activities, allows teachers to formulate policy. In 
addition, teachers may develop and co-teach university courses and prepare materials for 
student teachers, working as equals with university faculty and joining them in conference 
presentations about the project at local, state and national conferences. 

Whereas in-service programs typically address the interests of a school’s entire staff, 
CTEI offers teachers subject-specific seminars on such topics as state standards, methods, and 
curriculum. Led by a university faculty member, the seminars respond to identified interests 
and needs of the teachers. University personnel also help teachers to write grant proposals for 
pedagogical innovations, curriculum development, and summer activities. 

Teachers at the partner schools receive user privileges at the University of California, 
Riverside, Library. In collaboration with the library, project staff train teachers in information 
technology and support them in its use and assist professional development schools in gaining 
access to the Internet. The university gives each school a high-end computer and helps it to 
acquire additional refurbished computers from corporate donors. 

CTEI offers teachers who engage in classroom research the opportunity to consult with 
university faculty. In turn, professional development schools serve as research sites for School 
of Education faculty. 
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Evaluation and Project Impact 



Ongoing fomiative evaluation assesses student teachers and this leads to program 
development and revision. CTEI staff use information from videos, journals, portfolios, high 
school student evaluations of courses and prospective teachers, classroom observations, formal 
questionnaires, interviews, retreats, and reunions for graduates. 

During the period of FIPSE funding, external evaluators compared the experiences of 
CTEI student teachers with those of graduates of other southern California programs. They 
administered a general survey about student teaching experiences to first- and second-year 
teachers in 18 school districts and to student teachers in the professional development schools. 

The evaluators collected information about CTEI schools through site visits, year-end 
surveys, and interviews of school administrators, university supervisors, and cooperating 
teachers. They assembled data about completion rates and professional status after graduation 
and administered Teaching Tasks and Efficiency Ratings questionnaires. 

CTEI seems to have provided substantial benefits. CTEI students reported significantly 
more camaraderie and support than the teachers who had undergone traditional student 
teaching experiences. They felt more a part of the school and of the faculty and rated their 
preparation program higher than the comparison group. 

During the school year, student teachers showed significant gains in their ability to 
perfonn teaching tasks as well as the usual substantial decline in commitment to teaching 
which accompanies the dawning of a more realistic attitude toward their profession. CTEI 
students felt well prepared for their duties, more involved in self assessment and closer to 
students than the comparison group. Because they were at the school for an entire year, 
project participants were able to base their lessons not only on the curriculum but also on their 
knowledge of the students. Overall, the evaluators concluded that CTEI student teachers had a 
more authentic experience than students from traditional programs and entered the profession 
as if they were second year teachers. 





Ninety-four percent of the students graduated, and 80 percent went into teaching. In 
the two following years, 92 and 87 percent of CTEI graduates accepted teaching positions, 
compared with national rates of between 60 and 70 percent. 

Preliminary results of research conducted after the end of the FIPSE grant indicate that 
CTEI graduates feel well prepared for teaching. They attribute their confidence to the 
full-year teaching experience and to working closely with many teachers. The majority of 
graduates from traditional programs included in the research, on the other hand, do not feel 
well prepared, and believe that they would have been most helped by a longer and more 
realistic teaching experience and more support and collaboration— the very strengths of the 
CTEI approach. 

The evaluators found that the project did enhance the professional development of 
experienced teachers. Because teachers proposed many of the project’s activities and chose 
which ones to participate in, and because teachers from different schools planned and 
participated in activities together, they reported a breakdown of isolation, a willingness to try 
new ideas in the classroom and an appreciation for the collaborative, supportive environment 
created by CTEI. Teachers’ involvement in research further encouraged their professional 
growth and instructional innovation and signaled to them that research is not limited to 
university settings. Many teachers felt a sense of renewal in their work. 

These outcomes were greatest at the sites where implementation of all components took 
place. Teachers at one site, which had difficulty adjusting the master schedule for all 
cooperating teachers, did not exhibit the same level of collegiality as the others. On the other 
hand, the pilot site, which was in its seventh year at the time of the evaluation, reported the 
highest level of support and camaraderie among teachers, possibly because these effects 
increase with time. 
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Lessons Learned 



Because not all program components were implemented to the same extent at all sites, 
student teachers experienced uneven levels of support. It seems that certain general conditions 
influence the effectiveness of the CTEI experience: the desirability and location of the on-site 
meeting room, the extent to which the program becomes a part of the school culture, and the 
scheduling of the daily seminar. 

In the traditional model of student teaching, in which a wide pool of candidates for 
cooperating teacher assignments is available, it is possible to select individuals with considerable 
expertise. Because programs such as CTEI confine their pool of cooperating teachers to the partner 
schools, it is important to select professional development school sites with particularly expert 
teachers and to increase the expertise of cooperating teachers at the existing sites. 

Project Continuation 

CTEI has become the mainstream student teaching program at the university. 
Dissemination and Recognition 

The project received the Quality of Education Award from the California Council on 
the Education of Teachers, the Exemplary Teacher Education Program Award from the 
National Education Association, and the 1997 Distinguished Program in Teacher Education 
Award from the Association of Teacher Educators, which honors teacher education programs 
that exemplify collaboration between local schools and institutions of higher education. 

Cooperating teachers participated in grant-writing workshops and submitted proposals, 
individually or in collaboration with CTEI faculty, to funding agencies. Two teachers obtained 
grants from the MacArthur Academic Alliance, and another collaborated on a planning grant on 
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mathematics and science teaching from the Eisenhower State Grant Program. A team of 
teachers and CTEI faculty obtained funding from the University of California Office of the 
President to research professional development for practicing teachers. CTEI also received 
funding from the California Department of Education. 

Project staff, cooperating teachers and student teachers jointly presented the project at 
many local, state, and national meetings. CTEI continues to collaborate with the National 
Center for Innovation’s Teacher Education Initiative and to consult with a variety of 
institutions and educators working to establish professional development schools throughout 
the country. 

The program now includes five schools in three districts— three high schools, a middle 
school, and a newly-added elementary school. Because some of these are feeder schools, it 
will be possible to institute some innovative practices across sites. The project may expand to 
additional elementary and secondary schools next year. 

Available Information 



Further information may be obtained from: 

Judith H. Sandholtz 
School of Education 
University of California, Riverside 
Riverside, CA 92521-0128 
Telephone: 909-787-5798 
e-mail: judith.sandholtz@ucr.edu 
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PRESCOTT COLLEGE 
Center for Indian Bilingual Teacher Education 



Purpose 

The conviction that Indian children deserve Indian teachers moved members of the 
Tohono O’Odham Nation to approach Prescott College to help train Native American teachers 
for reservation schools. 

A decade ago, few Indian children were taught by members of their own ethnic group. 
Only 12 percent of certified teachers on Tohono O’Odham Nation were Indian. On the Hopi 
and Navajo reservations, the proportion of Indian teachers was 25 percent and 30 percent 
respectively. 

This not only meant that most Indian children were taught by teachers who were not 
familiar with their culture and who may also have believed them to be categorically in need of 
remediation, but for the many children in Arizona who speak only their native languages, 
communicating with their teachers was difficult if not impossible. To make matters worse, 
non-Indian teachers usually moved out of the reservation after one to two years, taking with 
them the benefit of their experience and leaving their classes to adjust to a new teacher all over 
again. The Tohono O’Odham leaders believed that this situation was largely responsible for 
Native American students’ dropping out of school at rates higher than those of any other ethnic 
group. 

Innovative Features 



In response to these concerns, Prescott College established the Center for Indian Bilingual 
Teacher Education (CIBTE), an adaptation of the college’s existing Adult Degree Program. The 
center serves adult Indians, living on mral Indian nations, who have already accumulated some 
community college credits. CIBTE leads to a B. A. degree with Arizona state credentials in 
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secondary, elementary, or special education, and endorsements in English as a Second 
Language (ESL) and in bilingual education. The latter means that graduates have been trained 
to use their native language in the classroom. 

CIBTE serves students in their community, where they take tutorials or work in small 
cooperative peer groups with a local mentor recruited by the college. Campus activities 
include a weekend orientation and a liberal arts seminar. The project provides an admissions 
counselor to help students interact with various college offices and to interpret the college’s 
requirements. In addition, each CIBTE student has an advisor with whom he or she 
establishes a personal relationship. The advisor contacts the student regularly, persisting even 
when the student does not return phone calls. Counselors, advisors, and project staff as a 
whole strive to communicate unconditional acceptance, caring and support to students. 

Each student’s program is individualized. Credits from other institutions are organized 
and applied to the degree, and students can challenge courses by assembling a life-experience 
portfolio documenting college-level learning. They then receive a study program carefully 
tailored to take into account past credits and experience and the college’s requirements. At 
periodic intervals, staff review these programs with students and chart their progress toward 
the degree. 

CIBTE adopts a bicultural approach to educating Native Americans. First and 
foremost, students are accepted and valued as they are. However, CIBTE also helps them to 
cultivate traits and skills for succeeding in the majority culture, such as assertiveness, 
individual initiative, and the ability to cope with paperwork. To encourage students to work 
independently CIBTE asks them to formulate a study contract for each course, meet 
mathematics and writing competency requirements, and write a graduation proposal 
synthesizing their learning. 

The college cultivates relationships with local school principals and superintendents. 
Directors of tribal education frequently come to campus to advise project staff, who in turn 
regularly visit tribal education offices. An advisory committee of tribal leaders and educators 
ensures that CIBTE continues to respond to Indian needs. 



Evaluation and Project Impact 



CIBTE began in 1988 with 15 students. By 1994, 97 had graduated from the program, 
21 of them with a bilingual or ESL endorsement. By 1998, the program had graduated 208 
students and served representatives of 20 tribes (a small minority have been non-Indians living 
and working on Indian nations) . All but one percent of graduates are teaching in Indian 
schools, whether on reservations or in rural communities near reservations. 

Perhaps the project’s most striking achievement has to do with retention, a major problem 
in Indian education. In 1988, for example, 77 percent of Native American students at the 
University of New Mexico dropped out, and even some tribal colleges document attrition as high as 
90 percent. In this context, CIBTE’ s average term-to-tenn retention of 69 percent (rising to 75 
percent in 1997) represents remarkable progress, as does the finding that ten percent of students 
graduating through 1996 are enrolled in or have completed M. A. programs. 

Lessons Learned 



When CIBTE was first conceived, its creators believed that Prescott’s orientation, with its focus 
on the individual, its competency-based requirements and its tradition of valuing diversity would be 
especially compatible with Indian cultural norms. However, it soon became apparent that the self-direction 
needed to complete such tasks as learning contracts, to fulfill mathematics and writing competency 
requirements, and to write a graduation proposal placed overwhelming pressures on some students, who 
were accustomed to the Native American focus on community. Nevertheless, staff discovered that these 
students can develop individual initiative given the right kind of support. This means that staff frequaitly visit 
them at home, talk through their assignments stq? by stq^, and give them encouragement. The students 
who do best are those who find a compatible peer cooperation group or a particularly caring mentor. 

Tribal politics had a major impact on the project. CIBTE staff had to contend with changes 
of administration in the tribes, shifting requirements by tribal financial aid offices, and evolving 
views of what constitutes appropriate higher education for Indians. College personnel 
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found that gaining the trust and respect of tribal leaders took time, but the process was helped 
along by interactions with the president and dean, and eventually by the results of the project 
itself, especially the successful placement of graduates in reservation schools. 

From 1990 to 1992 CIBTE had an Indian director. The reaction to his departure was 
reflected in a temporary increase in attrition. In hindsight, it appears that the college could 
have benefited from an increased awareness of Native American mores, while the Indian staff 
in turn needed greater familiarity with the ways of a primarily white institution. 

Given the financially precarious circumstances of most CIBTE students, it is not 
surprising that enrollment follows the availability of tuition monies, and this in turn has 
affected tribal representation in the project. When Title Vn scholarships, which brought in 
students from a number of tribes, ended, the college began working with the Navajo Nation to 
negotiate financial aid. Navajo representation among students then grew to 70 percent. The 
college is currently attempting to increase diversity in tribal enrollment. 

Project Continuation 

The project has been institutionalized, partly with the aid of other grant funds. It is 
administered by a program dean, two full-time faculty (Navajo and Apache), an admissions 
counselor, an assistant to the dean for operations, and an administrative assistant in CIBTE’s 
newly established sub-office in Window Rock, Ariz., on the Navajo Nation. 

CIBTE has grown to include post baccalaureate students seeking teacher certification, 
and students working toward B.A. degrees in teacher education, and counseling psychology 
and human services. 

In 1993 Prescott inaugurated “Culture, Environment and Education,” a six-month 
curriculum for Latino and Indian teacher education students in southern Arizona. The 
curriculum combines teaching methodology and environmental education— with a focus on 
local environmental issues— and has attracted a number of CIBTE students. 
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In 1997, CIBTE became part of Prescott’s Adult Degree Programs (ADP). Due to 
administrative changes and lack of funds, enrollment declined from a high of 157 Indian 
enrollments in 1995 to an average of 108 from 1996 through 1998. Approximately 70 percent 
of the students are Navajo. 

The Tohono O’Odham Nation currently supports scholarships for a four-year bachelor 
of arts program. This unique program allows students without prior college credits to 
complete a four-year degree and meet the community’s needs. As of February 1999, the ADP 
Tohono O’Odham Psychological Services (TOPS) program graduated five students (one 
student dropped out). 

Dissemination and Recognition 

With funding from the Arizona Humanities Council, CIBTE organized a statewide 
conference entitled “Native American Voices: Culture and Learning.” The gathering, which 
attracted 250 participants, featured Indian educators who presented traditional ways of 
learning. 

In addition to FIPSE funding, between 1989 and 1995 CIBTE received support for 
scholarships and operations totaling $1,310,800. Contributors included Title Vn, the Hearst 
Foundation, the Education Foundation of America, the Navajo Ford Consortium, Navajo Title 
V, the Ittleson Foundation, the Stocker Foundation, the Rockefeller Brothers Fund, and the 
Environmental Education and Training Foundation. 



199 





Available Information 



Further information may be obtained from: 

Vicky Young 

Coordinator for CIBTE and Native American Students 

Prescott College 

CIBTE/ ADP 

220 Grove 

Prescott, AZ 86301 

Telephone: 520-776-7116 ext. 3006 

Telephone: 520-628-6343 ext. 3006 

e-mail: vyoung@mail.prescott.edu 
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VANDERBILT UNIVERSITY 

Improving Undergraduate Teacher Education with Technology and Case-Based 

Instruction 



Purpose 

The courses that future teachers take in college do not always prepare them for the 
realities of the classroom. Faculty at Vanderbilt’s Peabody College noticed that students left 
reading courses with information about the factors affecting reading and knowing how to 
evaluate different instructional models, yet they seemed inflexible in their thinking about the 
problems encountered in teaching. They also appeared limited in their application of 
intervention techniques to new contexts. Their repertoire of alternative techniques to use when 
instruction did not go as expected appeared small. 

Teaching is a complex cognitive skill. Because it occurs in ill-defined environments, it 
requires not only knowledge about what to do but the ability to determine when and how to do 
it. Methods courses often fail to help prospective teachers because they teach procedures and 
neglect problem-solving skills. Lecture methodologies and written or videotaped case studies 
that oversimplify complex problems are especially ineffective. 

Lectures do not work because they do not teach students to analyze the effects of 
situational and classroom contexts on a teacher’s choice and application of method, since it is 
impossible to describe all the nuances and dynamics of classroom interactions. Because of the 
linear nature of lectures, examples used to^ explain concepts can seem unrelated to each other 
and students have to struggle to make sens^of disparate experiences. 

Nor is all case-based instruction successful. In written case studies, often problems are 
too narrowly described to show how multiple factors interact to create problems. Videotaped 
cases are of necessity fragmented and do not allow for comparisons across cases. Because 
videotape only allows the instructor to run through an entire segment once rather than 
reviewing specific scenes as often as needed, videotaped cases do not provide opportunities for 
exploration and problem identification at the level required by case-based pedagogy. 
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For a number of reasons outlined below, project faculty concluded that a 
videodisc-based problem solving approach would help students to develop flexible thinking 
about problems and the spontaneous use of strategies to solve them. 

Innovative Features 



During the course of the project, faculty developed, tested and disseminated 16 
videodisc-based cases enhanced by HyperCard and microcomputer technology, to be used in 
developmental and reading methods courses. They adapted the case-based procedures used in 
medicine, law and business, which have found case-based instruction useful in problem-solving 
and preparing students to know and act simultaneously. The resulting discs portray several 
units of instruction and present teacher and student interviews, numerous classroom scenes, 
procedural activities, and learning among various groups. 

Faculty chose reading education courses for videodisc enhancement because they are 
vital to the national interest. They believed that reading education courses, which all 
undergraduate teacher education students are required to take, are especially suited to 
problem-centered instruction developed around case studies. To counteract fragmented 
learning they developed cases to teach across both developmental and remedial reading. 

They chose videodiscs because these discs allow rapid random access and 
frame-accurate searches and freeze-frame and menu-driven control for cross-referencing of 
information across cases. In addition, it is possible through computer links to incorporate 
additional perspectives using text, sound and graphics. Videodiscs provide such information as 
teachers’ nonverbal cues and simultaneously-occurring classroom events, which students can 
reexamine and cross-examine from multiple perspectives. Being able to see the events that 
lead to teachers’ decisions allows the transfer of knowledge to real-world situations. In short, 
videodiscs, especially when linked to computers, encompass ail the possibilities of print and 
videotape while avoiding their shortcomings. 



O 



202 



To complement the videodiscs, project faculty created supporting HyperCard 
technology, lesson plans, and other materials. HyperCard technology enhances the 
effectiveness of instruction by providing students access to multiple sources of information 
about each case. 



Evaluation and Project Impact 



In the course of the project, faculty used pretest and posttest control groups to evaluate 
students’ perception of changes in knowledge and the usefulness of various methods of 
instruction. They evaluated two sections of developmental and remedial reading methods 
courses, one taught with videodisc, case-based instruction and one with traditional instruction, 
including lectures with readings, written cases, and demonstration videotapes. They also 
compiled case studies of students who were taught with the videodisc methodology. The 
evaluations comprised many measures, from paper and pencil tasks to structured interviews. 

Faculty measured the effects of the videodisc methodology on students’ participation in case 
analyses in the college classroom, their knowledge of factors contributing to literacy development, 
their knowledge and application of instructional procedures in the school classroom, and their use 
of decision-making strategies when confronted with problems during teaching. 

Preliminary data indicate that student knowledge increased during the course and that 
students changed their perception of the value of certain teaching methods, i.e., more felt that 
class discussions and field experiences were valuable ways of learning and that 
problem-solving abilities were enhanced by the use of video. 

In the classes that used videodiscs there was abundant discussion, with students 
showing more factual knowledge and ability to analyze problems from multiple perspectives 
and to use infonnation to solve new problems. These students asked more questions and made 
more comments, and their questions showed their ability to evaluate and synthesize the 
material. They showed an increased ability to consider alternative solutions to problems, to 
adjust appropriately instructional routines and procedures, and to Justify decisions. 
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Faculty observed that students entered the course with a limited understanding of the 
complexities of reading instruction, focusing on narrow aspects of reading development to 
guide their analysis of the initial video case. Eventually, they improved in their ability to 
adopt additional perspectives for thinking about classroom situations, something which, 
according to researchers, does not normally happen until after students begin teaching. 

To compensate for the small size of the original evaluation sample, the effectiveness of 
the videodiscs was tested at other institutions, where data were gathered with surveys and 
structured interviews. These follow-up tests revealed similar results: students participated 
more in class discussions; students adopted new perspectives to analyze teaching dilemmas; 
and students increased their abilities to identify problems and offer solutions. 

Project Continuation 

At Vanderbilt, the cases are used every semester in several sections of developmental 
and remedial classes. They are also used on a continuous basis in at least six other sites. 

Dissemination and Recognition 

The project was the subject of eight articles (one of which received the Distinguished 
Research in Teacher Education Award from the Association of Teacher Educators), eight 
technical reports, and 25 presentations. It was visited by faculty from the United States and 
abroad. 

The videodisc material has been converted to CD-ROM and published as Risko, V, J,, 
& Kinzer, C, K,, Multimedia Cases in Reading Education (McGraw-Hill, 1999), 
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Available Information 



Further information may be obtained from: 

Victoria J. Risko 
Charles K. Kinzer 

Peabody College of Vanderbilt University 

Nashville, TN 37203 

Telephone: 615-322-4596 

e-mail: Victoria.J.Risko@Vanderbilt.edu 

Telephone: 615-322-8101 

e-mail: Charles.K.Kin 2 er@Vanderbilt.edu 
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Vm. PERFORMANCE FUNDING AND REPORTING 
IN COLLEGES AND UNIVERSITIES 
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INTRODUCTION TO PERFORMANCE FUNDING 
Robert B. Stein 

Missouri Coordinating Board of Education 



Funding perfonnance is not a new concept for academicians, as any faculty member 
who has engaged in discussions of merit pay will testify. Funding linked to group rather than 
to individual performance is, however, a relatively recent phenomenon that has drawn the 
attention of policy makers, legislators, and governors as states seek new ways to make 
educational policy. 

While performance funding initiatives vary in shape, size, and scope, they all involve 
rewards for outcomes. Performance funding is based on the assumption that previously 
established priorities will drive funding policies and that, by rewarding results, individuals, 
groups, or entire institutions will be motivated to produce desired changes within the system. 

Burke and Serban (1997) report increased state interest in using some form of 
performance funding as a strategy for improving the quality and accountability of higher 
education systems. The design and implementation of performance funding programs that 
highlight teaching and learning issues have become major challenges as states seek to balance 
legislative interest in productivity and efficiency measures with an emphasis on quality of 
student outcomes. 

Missouri’s Funding for Results initiative supports enhanced quality and accountability 
in the state’s public higher education system. With support from FIPSE, a unique 
performance funding program is evolving in Missouri. While its use in other states will 
require adaptation, the lessons learned in Missouri will be helpful to institutions and states 
intending to use performance funding as an innovative strategy to bring about improvements in 
teaching and learning. 

Burke, J.C. and Serban, A.M. (1997). Performance Funding and Budgeting for Public 
Higher Education: Current Status and Future Prospects. Albany, N.Y.: Rockefeller Institute 
of Government. 
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MISSOURI COORDINATING BOARD FOR HIGHER EDUCATION 

Funding for Results 



Purpose 

While a commitment to quality teaching and learning is included in the rhetoric of most 
state-level policymakers, state appropriations for higher education have historically been driven 
by more easily measurable factors, such as inflation or credit hours generated. As a result, 
value is placed on growth at the expense of continuous quality improvement. Innovations in 
teaching and learning are often left to individual faculty initiatives rather than included in 
budgets. Missouri chose to confront this imbalance by redesigning its funding policies to 
emphasize results in specific areas with quantifiable goals. 

Based on previous planning priorities. Funding for Results (FFR) recognizes institutions for 
their achievement of quality goals and for their design and implementation of faculty-driven 
teaching and learning improvement projeas. As a statewide initiative, FFR is currently working 
with 18 public two-year campuses and 13 public four-year campuses. Collectively, there is a 
potential to eventually affect 8,083 full-time equivalent (FTE) faculty and 131 ,240 FTE students. 
Tliroiigh a collaborative approach, Missouri has started a process that engages faculty, 
administrators, legislators, and business leaders in a new partnership emphasizing results. 

Innovative Features 



Missouri’s approach to performance funding includes several innovative features 
designed to break down barriers, overcome resistance, and infuse new understanding, interest, 
and excitement in the business of quality improvement. 

In Missouri, budget requests are designed so that each institution has a core budget that is 
carried forward each year; new dollars which support major public policy initiatives are requested 
in addition to the core budget. The FFR dollars earned by an institution in a given year are placed 
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in that institution’s core budget and thereby are retained in succeeding years. In this way, 
although the total amount of money appropriated in any given year is relatively small, i.e., 
less than two percent of an institution’s total budget, its impact is magnified. 

Too often a heavy hand from a legislature or a central administration causes a state to 
design agendas linked to budget policies. For example, headcount formulas may be used to 
reward institutions for increased numbers of students who meet specific standards, or a 
particular assessment instrument may be mandated to evaluate student performance regardless 
of the type of institution involved. These approaches often pit one state institution against 
another, thus reinforcing undesirable competition for scarce resources and causing 
fragmentation of the higher education system. In response, faculty and administrators perceive 
unfair treatment and claim that differences in mission, e.g., clientele served and programmatic 
mix, have not been acknowledged. 

FFR seeks to acknowledge institutional differences and to allow flexibility in the 
specification of measures while motivating institutions to work toward common state goals. A 
major component of FFR involves the design and implementation of campus teaching and 
learning improvement projects at each of Missouri’s public colleges and universities. These 
projects require new partnerships between faculty and administration in the use of performance 
funding incentives provided by the state. Broad-based parameters, such as use of a database 
indicator system, extensive faculty ownership, systematic evaluation, and incentive systems, 
were established for eligibility to participate in this program. Institutions are, however, given 
great flexibility in the design of particular projects, which range from improving the basic 
skills of entering students to enhancing the residency experiences of doctoral students. 

FFR acknowledges differences in institutional mission while rewarding each institution 
for its contribution to meeting a state need. Institutions are also being encouraged to become 
more distinctive. In each case, performance measures are identified so that achievement of 
institutional mission can be demonstrated and rewarded in the context of a unified plan for 
public higher education in the state. 
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Because FFR is using data with common definitions, the state is well positioned to 
provide annual reports that compare institutions on statewide goals, e.g., six-year graduation 
rates, as well as to describe some unique contributions from each institution concerning 
teaching and learning improvements. 

The faculty is generally perceived to have primary allegiance to disciplines and 
departments rather than institutions. While faculty have common interests and concerns 
regarding student learning, their voices are too often heard only in reaction to, rather than 
during the creation of, statewide policies. Involving faculty in discussions with state-level 
policymakers requires the design of new expectations that circumvent traditional hierarchical 
stnictures and foster direct lines of communication. In response to these challenges, Missouri 
chose to create an FFR Committee on Teaching and Learning which consists of faculty only. 
This group has the major responsibility for providing reactions to state policymakers on FFR 
and its effect on teaching and learning. 

Traditional reward structures associated with teaching and learning improvements are 
often designed for individuals, e.g., merit pay systems and faculty awards. These approaches 
to perfonnance funding, as conceived by FFR, are narrow and restrictive. Instead, FFR 
promotes rewards to departments, colleges, deans’ offices, and the campus as a whole by 
working with the institution to redesign its approach to the flow of money. Rewards for 
teaching and learning improvements are slowly becoming integrated into the campus culture, 
as institutions are encouraged to set goals, establish appropriate units of analyses, and then, 
once goals have been met, to release rewards. As a part of FFR, a competition for the “Best 
of the Best” campus teaching and learning projects was designed; faculty established the 
criteria and evaluated the applications; and the winners were acknowledged at a Governor’s 
Conference on Higher Education. 
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Evaluation 



The FFR initiative is inherently linked to institutional assessment programs and the design 
of statewide performance indicators. Institutions annually submit data for statewide analysis on all 
FFR elements. Missouri then reports on its progress toward its public policy goals, including those 
associated with FFR. In addition, institutions submit an annual accountability report concerning 
FFR dollars linked to campus teaching and learning improvement projects. 

During the life of the FIPSE grant, external evaluators made annual site visits to 
Missouri, met with stakeholders, and submitted a report of their findings. During the second 
year of the grant, a questionnaire measuring the level of familiarity and support for FFR was 
distributed to all full-time faculty. Acknowledgments, comments, and inquiries by external 
audiences concerning FFR are being tracked as an additional source of data. 

Project Impact 

Missouri’s public colleges and universities are working together to ensure access, quality, 
efficiency, and an overall improvement in the state’s system of higher education. While FFR is still in 
its infancy and financial support continues to change, the slate has reaffinned its commitment to perfonnance 
funding as a strategy for change, and FFR has become more coherent as a major public policy initiative. 

To date, positive results associated with FFR include increases in the assessment scores of 
graduates, both in general education (72 percent to 76 percent for baccalaureate degree students; 19 
percent to 81 percent for associate degree students) and in the major (63 percent to 83 percent for 
baccalaureate degree students). There have been increases in pass scores for baccalaureate students 
on licensure, certification, or registration examinations (83 percent to 87 percent for baccalaureate 
degree students, 82 to 96 percent for associate degree students); increases in the six-year 
graduation rate for public four-year institutions from 47 percent to 48 percent; and increases in 
minority graduation as a percentage of the graduating class for baccalaureate students from 
seven percent to ten percent. The number of community college students transferring 



successfully to four-year institutions has increased 16 percent, and community college 
vocational job placements have increased from 69 percent to 76 percent. While the causal 
relationship between FFR and these achievements is not conclusive, FFR is clearly responsible 
for the identification of priorities for funding, for the establishment of assessment measures, 
and for helping institutions to accept that part of their state allocation is linked to results. 

In addition, as a result of FFR, all 31 public campuses have chosen to participate in the 
design and implementation of faculty-driven teaching and learning improvement projects. 

While the long-tenn effect of this initiative is not known, initial results indicate an 
increased interest and enthusiasm from faculty for this effort (at one university, during the first 
year alone 69 proposals outlining new teaching and learning initiatives were submitted). 
Institutions’ accountability reports suggest some preliminary findings. For example, greater 
retention, higher GPAs, and increased satisfaction result when students are involved in 
learning communities; at-risk students experience higher completion rates and increased 
retention (from 68 to 82 percent) when participating in specialized instruction that emphasizes 
subject matter and individual needs; and mean ACT composite scores increase for students 
involved in critical thinking projects. 

Between FY 1994 and FY 1999, Missouri’s public four-year institutions received an 
addition to their base budget of $42,098,768 from FFR, representing 16.7 percent of new 
money allocated. Included in this amount is a $2.7 million annual allocation for teaching and 
learning improvement projects. Public two-year institutions began to receive FFR money in 
FY 1995. Through FY 1999, their total of $7,174,671 represents 13.4 percent of new money, 
of which $1.4 million is dedicated annually to teaching and learning improvement projects. 
Presidents and chancellors of Missouri’s public institutions have expressed their support for 
allocating one percent of each institution’s operating budget to these projects. 

Initially, the concept of perfonnance funding was widely questioned. Some educational 
leaders believed that increased financial support should be considered an entitlement associated 
with inflation. FFR was also resisted as an intrusion into the operation, management, and 
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structure of each institution. These perceptions changed during the project. In its second year 
32 percent of Missouri’s faculty at public institutions returned a survey on FFR. More than a 
third of those responding indicated that they were familiar with FFR and, after reading a fact 
sheet, over 67 percent indicated support for its goals. 

Lessons Learned 



Initially, several administrative representatives were critical of the FFR initiative and 
suggested that it not be officially launched until all questions could be answered and problems 
resolved to greater satisfaction. However, by maintaining its commitment to move ahead with 
performance funding even without a perfect model in place, Missouri was able to make 
substantial progress in a short time. The fact that FFR started with low financial stakes further 
helped to reduce anxiety and increase ownership. 

Making participation in the teaching and learning improvement aspect of FFR voluntary 
removed institutional defensiveness. The reward for participation was large enough to get the 
attention of all public campuses within the state, yet small enough to avoid criticism that FFR 
would be taking money away from other important agendas. By establishing broadly defined 
eligibility criteria, the common goal of teaching and learning improvement was promoted 
while, at the same time, flexibility was allowed in the design and implementation of particular 
teaching and learning improvement projects. 

Maintaining commitment in the face of adversity requires constant communication. Identity ing 
and addressing myths associated with performance funding early in the process by, for example, helping 
faculty and adrnimstrators see FFR as a strategy to promote and acknowledge results rather than as a 
bureaucratic demand reduced misunderstandings. Commitment to a common vision and the use of 
consistent language helped to increase support across diverse groups. By initially building 
identification with the overall goals and benefits of a performance funding system related to 
institutional missions and goals, different opinions about choice and definition of data elements and 
allocation structures could be explored without threatening to collapse the initiative. 




" 210 
51 $ 



Perceptions of a new initiative are inherently linked with images of additional work and 
time commitments which will compete with established agendas. FFR was often viewed as an 
add-on to existing initiatives and, as a result, was initially pushed aside with a promise to be 
given attention at a later time. Promotion of FFR as a new approach that should be integrated 
into planning initiatives rather than as a new activity helped to shift the emphasis to a redesign 
of existing activities. 

In systemic statewide change efforts, faculty voices are often not heard early enough in 
the process. By identifying improvement in teaching and learning as a central tenet of 
statewide policy reform, designing a structure that requires faculty participation, and fostering 
communication among faculty across departmental, college, and institutional boundaries, FFR 
captured faculty interest, excitement, and involvement early in the change process. 

While the lack of explicit assumptions in the first year of FFR created some 
discomfort, this had the unintended effect of allowing differences of opinion to coexist as 
faculty and institutions gained experience with this new approach. As the tension increased 
between opposing positions, e.g., evaluation of student outcomes vs. program perfonnance as 
an appropriate unit of analysis, differences of opinion were resolved by the adoption of explicit 
assumptions. As FFR continues to evolve, these assumptions will help to guide the review and 
revision of this initiative. 

Colleges and universities often initiate strategic planning, design assessment programs, 
and recommend budgets without understanding the importance of integrating these three 
activities. FFR has been used as a unifying concept by which each of these activities becomes 
interdependent. Funding of higher education should not exist in a vacuum; it should be 
driven by previous planning priorities and assessment results. By aligning these three 
activities, requests for additional resources do not emerge sporadically but are directly linked 
to planning priorities and results. 
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Inherent in FFR is a commitment to systematic measurement. At first, many of the 
FFR elements placed an emphasis on demonstrating increased activity, e.g., increasing the 
number of students participating in assessments. The introduction of quality measures has 
helped to shift the emphasis. For example, institutions no longer are rewarded for merely 
doing assessment but for how students perform on particular assessments. 

FFR uses a finite list of measurements to determine particular budget recommendations 
based on perfonnance. The difficulty of identifying key perfonnance indicators and then 
prioritizing funding elements was not anticipated. Measurement sends a powerful message 
about what is perceived as important. Suddenly, colleges and universities were no longer 
criticizing measurement systems by citing imperfections and limitations but, rather, were 
calling for a massive data system to ensure that every activity of importance was measured and 
potentially rewarded. One challenge to FFR is to help institutions understand that massive 
data systems have, limited use in informing policy discussions. 

The ability to bring about systemic change requires more than genuine interest and 
conceptual commitment. The magnitude of effort required when working with total systems 
can become overwhelming. Designing an open process in support of new policy formulation 
and development, mediating different perspectives among diverse groups, and maintaining an 
appropriate balance between discussion and action demand a keen sense of timing, an ability to 
take, risks, and an appreciation of traditions. 

Organizational decisions are often made without the internal staffs full understanding 
or support. Time was allotted during the first year of the project to make site visits to 
campuses throughout the state, but project staff failed to anticipate the amount of time required 
to explain and to build support with key internal staff in the agency. The importance of 
horizontal communication with peers and of building a solid support base from within cannot 
be overstated. 
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Higher education faculty and administrators tend to spend inordinate amounts of time dissecting 
and evaluating ideas through extensive examination, explanation, and critique. To capture the attention 
and maintain the interest and support of the public, legislators, and the governor, on the other hand, 
requires consistent, concise, and convincing messages. Project staff initially underestimated the 
importance of redesigning communication styles and presenting content clearly for external audiences. 

Missouri’s commitment to redesign budget policies to include FFR raised the stakes 
associated with data reports. In the early years of the project, interest in accountability 
directed overwhelming attention to external audiences, at the expense of internal audiences. 
This imbalance reinforced compliance, especially at heavily regulated institutions. Missouri is 
expending greater effort than was originally anticipated at integrating accountability and 
improvement initiatives to increase faculty involvement, interest, and responsibility. 

Changes that withstand the test of time inevitably face challenges from emerging 
leaders who are motivated to establish credibility, influence, and impact. When 
administrations turn over, the tendency to associate programs with individuals rather than with 
systems can hanu existing initiatives. To ensure that reform efforts remain apolitical, 
bipartisan support must be garnered, nurtured, and maintained. 

Project Continuation and Reco2nition 

Recent discussions by presidents and chancellors from public two- and four-year 
institutions indicate general agreement that, while FFR should be reviewed and improved, the 
basic approach is working. In February 1998, the Missouri Coordinating Board for Higher 
Education reaffimied its support for FFR and established a set of guiding principles to help ' 
shape any future revisions to its perfonuance funding policy. 

External consultants identified several strengths of FFR and indicated that Missouri’s 
emphasis on teaching and learning could provide a national model for good practice. The state’s 
successes in continuing to build campus ownership, in promoting networking to improve teaching 
and learning, and in using FFR as a catalyst for change were considered noteworthy. 



ERIC 




219 



Missouri has been challenged to improve its communication with a larger group of 
legislators, to develop a more rigorous and standardized assessment of teaching and learning 
improvement projects, to create a better alignment with new state initiatives, and to monitor 
the long-term effect of FFR on institutional budgets. 

FFR was featured as a case study during the 1997 Education Commission of the States’ 
National Conference. It was also referenced in “Grading Colleges Requires More Than Math,” 
an article in the November 9, 1997, “Week in Review” section of the New York Times. 
According to one national expert on performance funding, Missouri’s program is evolving so 
that it “. . . can show the country how to make performance funding not just a passing 
phenomenon but a lasting success.” 

Available Information 



This project has generated a number of articles, brochures and reports. General 
information about the project and copies of published materials may be secured by writing or 
calling: 

Robert B. Stein 
Assistant Commissioner 

Missouri Coordinating Board for Higher Education 

3515 Amazonas Drive 

Jefferson City, Missouri 65109 

Telephone: 573-751-1794 

Fax: 573-526-5431 

e-mail : robert?cbhe400@admi n . mocbhe. go v 
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PRAIRIE VIEW A&M UNIVERSITY 

Developing Effective Performance Indicators for Different Learning Environments 
Purpose 

During the past few years, higher education found it increasingly necessary to explain 
and defend itself to a variety of internal and external constituencies. Such demands for 
assessment and accountability resulted in a growing trend toward the use of performance 
indicators. However, there existed no model or process for the regular and public assessment 
of the degree to which a university and its agencies were achieving the state’s goals and 
fulfilling priorities. The primary purpose of this project, using three campuses of the Texas 
A&M University System (TAMUS), was to develop and disseminate such a model and 
process, along with a list of valid, explicit, quantifiable performance indicators suitable for 
longitudinal analysis of achievement and productivity. 

Innovative Features 



It was agreed at the onset of planning that any accountability system developed must 
champion the cause of multiple missions and advocate the principle that differences in mission 
do not imply differences in quality. By recognizing and advocating a variety of emphases in 
mission the TAMUS model responded to individual campus missions and rejected the 
traditional conflict in priorities among teaching, research, and service. Accompanied by a core of 
indicator activities that all institutions must perform, the model contributes to the accountability that 
flows from clarity, fulfillment, and measurement of individual institutional mission. 

In early 1996, TAMUS set about to assess performance within its ten universities and 
eight agencies by developing an accountability system that included reporting periodically on the 
results achieved in relation to goals assigned and resources received from its public benefactors. 
Guided by the principle that consensus and shared ownership are crucial in developing 
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perfonnance measures, the chancellor initiated a participatory process for developing the core 
of indicators that would apply systemwide, researching measures to ensure their validity, 
providing options for individual institutions and agencies, and involving faculty leaders. 

Four systemwide task forces were established. Membership from the universities and 
agencies included faculty senate leaders, senior- and mid-level administrative staff, from chief 
executive officers to student affairs, and lay members from advisory groups within TAMUS. 
The charges for the four task forces were as follows: develop a TAMUS response to changing 
state needs, develop or refine processes, as needed, to ensure effective participation by 
external interests in targeting teaching, research, and service, develop a list of valid, explicit, 
quantifiable performance measures suitable for longitudinal analysis of achievement and 
productivity, and develop guidelines for financial incentives that are linked to performance 
measures. 

The task force recommended the following key accountability areas and indicators for 
evaluating and improving academic quality in TAMUS. The task force also recommended that 
certain core indicators in these areas (indicated by asterisks in the list below) apply to all 
members. 

1 . Mission Focus and Fulfillment 

Development of an approved strategic plan to support the mission statement * 
Demonstrable attainment of goals contained in the approved plan* 

Institution’s or agency’s contribution to the state of Texas or region* 

Percentage of total dollar value to historically underutilized businesses* 
Institution’s or agency’s contribution to the state of Texas Higher Education 
Planning Benchmarks, 1997-2001 

2. Quality of Faculty 

Academic and other credentials of faculty 
Ratio of full-time faculty to full-time employees 
Accreditation of degree-granting programs 
Average annual salaries for full-time faculty by rank 
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3. Teaching Quality and Effectiveness 



Percentage of lower-division classes taught by tenured or tenure-track faculty* 
Percentage of students completing courses* 

Graduate or current student satisfaction with the instruction provided (for 
example, student opinion surveys and ‘‘consumer feedback”) 

Graduation rates of first-time, full-time, first -year students* 

First-year retention rates, broken down by student demographic groups* 

4. Student Success 

State licensure examination pass rate (for example, EXCET and PE exam) 
Employers’ satisfaction with graduates 

Number or percentage of graduates who have employment in their choice fields of 
graduation 

Degrees granted by levels 

Pass rate on Texas Academic Skills Program categorized by student demographic 
groups 

Percentage of graduates accepted to graduate and professional schools by the time 
of graduation 

5. Financial Responsibility and Efficiency 

Net total current fund revenues to total current fund revenues* 

Net auxiliary enterprise revenues to total auxiliary enterprise revenues* 
Administrative cost as a percentage of total institutional expenditures* 

Space utilization ratios of classrooms and laboratories* 

Education and general expenditures per full-time equivalent (FTE) student 
Education and general expenditures per credit hour 
Plant equity to plant debt (ratio) 

State appropriations as a percentage of total revenues 
Total private gifts as percentage of state appropriations 
Annual education and general expenditures per degree awarded 

6. Research Funding and Quality 

Annual amount of research dollar expenditures* 

External or sponsored research funds as a percentage of state appropriations 

Number of dollars in non-state, nonlocal research support 

Ratio of non-state-to-state research dollar expenditures 

Annual income from patents, licenses, agreements, and copyrights 

Annual research dollar expenditures generated per FTE graduate student 



O 



223 



7. Institution and Agency Cooperation and Collaboration 

Number of articulation agreements with community colleges 

Number of transfer students to and from the institution 

Number of collaborative projects with other state institutions and agencies 

Number of institutional partnerships with K-12 school districts 

8. Public Service 

Institution’s or agency’s level of service to Texas residents as measured by 
enrollment in extension and service programs (for example, distance education, 
arts, and health-related services) 

Public service hours of library (number of hours per week library open) 

Number and dollar amount of institution’s or agency’s resources, services, and 
expertise made available to meet a community need outside the context of the 
instructional and student services program 

* Indicators currently required of all state institutions by the Texas legislature 

Each TAMUS institution and agency first prioritizes the above eight accountability 
areas and then the mission-specific indicators. The task force recommended that the 
institutions and agencies limit their indicators to a total of 20. The primary philosophy is 
continuous performance improvement and value added through time, rather than performance 
comparison. The TAMUS institutions and agencies are currently working within these 
parameters to come up with their respective models. 



Evaluation 



In addition to the involvement of several system-wide committees, the ultimate evaluation of the 
model and the process will be its adoption, all or in part, by TAMUS and the Texas state government. 
Beginning in December of 1998, the Texas State Legislature required aU institutions to formally i^x)rt 
and comment on their performance for 1996, 1997, and 1998— the first of an ongoing set of hearings 
concerning results of the indicator model. At present, the effort is to secure more buy-in for the model 
among internal and external stakeholders, particulaily campuses in the TAMUS. 
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Numerous publications and presentations which flowed from the project and the 
reaction from such events gave the project validity, as did professional evaluation and 
acceptance of the results by a myriad of international, national, and regional forums and 
publications. However, the primary indicator of the project’s success, set out in the original 
proposal, was the general acceptance of the model by the state of Texas. 

Project Impact 

The movement toward developing a formal system of accountability constituted a 
pre-emptive strike designed to heighten the effectiveness of TAMUS and improve the public’s and 
policy makers’ understanding of and confidence in TAMUS and higher education in general. By 
developing such a pre-emptive strike, TAMUS managed to forestall possibly harsher legislative actions. 

Lessons Learned 

The number of performance indicators used in a given system probably should not 
exceed 20. The goal should be to produce a general institutional assessment rather than a 
precise measurement. Indicators are simply one of many tools available to monitor quality 
assurance, enhancement, efficiency, and effectiveness.* 

State-level or top-down models do not normally produce the constructive, substantive, 
and sustained results that campus-based and -initiated performance systems produce. 

Externally imposed systems are rarely accepted by internal stakeholders, such as faculty. The 
frequent result is resistance rather than reform. 

Two central constituencies essential to the success of such models are the faculty and 
the legislature. Both parties need to be involved in developing the model and need to buy into 
the processes. Change must be initiated by an institution’s administrative leadership. 
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The performance indicators proposed often lack validity and reliability. This 
shortcoming is particularly prevalent in top-down, externally imposed systems that have been 
hastily adopted, using what Ewell (1996) termed “legislation by fax.” The public and 
legislature seem enamored of certain indicators, such as small classes and senior faculty 
teaching lower-division courses, that are not necessarily effective in promoting learning. 

Capstone institutions will be limited somewhat in the qualitative improvement they can 
demonstrate with certain growth-oriented indicators, such as persistence rates and graduation 
rates. Rather than a value-added approach, benchmarking with a national sample of 
institutions, the approach employed by the State University of New York, may offer greater 
promise for demonstrating efficiency and productivity. 

A single indicator model cannot be applied to all types of institutions effectively 
without diminishing or distorting diverse missions and exacerbating the traditional conflict in 
priorities among teaching, research, and service. Flexibility is necessary to fit the mission of 
the institution or agency. 

Despite the current campus emphasis on using qualitative vehicles such as portfolios, in 
the case of state systems, performance indicators should almost always be quantitative in 
nature, because policymakers are dubious about results that cannot be audited, verified, or 
submitted to precise comparison. Furthermore, graduate education and agency performance, for 
example, are rarely incorporated into these models. Improving undergraduate education was 
paramount during the early development of such indicator systems, particularly top-down models, 
but the current /trend is toward ensuring better use of resources, and linking resources to outcomes. 

The indicators should have real financial incentives for institutions and agencies, and 
direct individual rewards for those directly responsible for success in meeting the goals. 

Finally, many of the accountability systems that have been ineffective failed to embrace 
or communicate in timely, understandable terms the results obtained from the performance 
indicator model. Innovative approaches such as a home page on the Web, a flier of the sort 
used by the Colorado System, and an insert for the Sunday edition of the major papers are 
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ways to get an institution’s message across. 
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Project Continuation 



The results of the project have been shared with a number of institutions and states. 
For fiscal year 1999-2000 the state of Texas has added some indicators to its statewide system 
and particularly focused on a specific set of indicators for athletics. It is unknown how 
performance indicators will be transformed into funding elements, but the project plans to 
compare its results to a set of national benchmarks, to gauge its progress on institutional goals 
in state agency and system planning efforts, and to gauge consumer satisfaction with the 
educational product. 



Available Information 



The project has generated three monographs, several articles, a set of published 
proceedings, and several national and one international presentation. General information 
about the project may be obtained by contacting the following: 

Jerry Gaither 

Prairie View A&M University 

Institutional Effectiveness, Research, and Analysis 

P.O. Box 65 

Prairie View, TX 77446 
Telephone: 409-857-2448 
Fax: 409-857-2451 
e-mail: gerald_gaither@pvamu.edu 

or 

Lee Blank 

Texas Ac&M University System 
309 Administration Bldg. 

College Station, TX 77843-1259 
Telephone: 409-845-6646 
Fax: 409-845-6358 
e-mail: ltb@adm.tamu.edu 

or 
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Vema Dewees 

The Texas A&M University System 
College Station, TX 77843-1138 
Telephone: 409-845-6459 
Fax: 409-845-8695 
e-mail: dewees@statehq.tamu.edu 

or 

Ray Hayes 

Texas A&M University - Corpus Christi 

6300 Ocean Drive 

Corpus Christi, TX 78412 

Telephone: 512-994-2321 

Fax: 512-994-5810 

e-mail: ray-hayes@tamucc.edu 




228 







IX. DISSEMINATING PROVEN REFORMS 
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“DISSEMINATING PROVEN REFORMS” 
AN OVERVIEW 
Robert E. Shoenberg 



“Disseminating Proven Reforms” is devoted exclusively to multiplying the effect of 
successful innovations. FIPSE has for many years made dissemination grants through the 
Comprehensive Program, with the grantee institutions using a variety of processes for 
broadcasting their successes. “Disseminating Proven Reforms” differs from those earlier 
efforts in that it is devoted exclusively to dissemination and employs a single mentoring model 
as the means of propagating programs. 

The Format 



The purpose of dissemination is to “grow” new programs, similar but not identical to 
the original, in places where they did not exist before. Disseminators often use agricultural 
analogies and terms like “growing,” “broadcasting,” and “propagating” to describe different 
approaches to their work. Making conference presentations and publishing materials, for 
example, might be compared to scattering seeds randomly over unprepared ground, where 
few, if any, will germinate, take root and thrive. “Disseminating Proven Reforms,” on the 
other hand, uses what could be described as a grafting strategy, attaching buds and branches to 
strong rootstock. While this method affects only a few plants, if the soil is right and the plants 
are well fed and watered they will grow and continue to propagate. Here the graft represents 
the innovation that has proven successful at the college or university where it was developed, 
and the rootstock symbolizes the institution committed to adapting the innovation. The soil 
represents the circumstances at the adapting institution; the water, the tutelage and 
encouragement of the mentor institution; and the fertilizer, sufficient funding to encourage the 
efforts of both mentor and adapters. 
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Dissemination has the fullest and most lasting results when the creators of demonstrably 
strong programs are brought into a one-on-one working relationship with those who are willing 
and ready to adapt their innovations. The association between mentor and adapter needs to be 
sustained (as opposed to a single consulting visit) and personal, with each party acknowledging 
its obligations to the other for the success of the process. 

“Disseminating Proven Reforms” was designed to embody these principles. The 13 
mentor programs were chosen on the basis of their efficacy as evidenced by appropriate 
outcome measures. Sometimes these were statistical measures, but not all programs lent 
themselves to such assessments, and those that did not were judged on the basis of more qualitative 
evidence. The applications to the competition came from mentor institutions who had already 
chosen and secured commitments from their adapters. The likelihood that the adapting institutions 
would prove fertile soil for dissemination was an important consideration in the selection process. 

The giants were for two years. The “Disseminating Proven Reforms” model called for 
conferences of mentors and ad^ters on the former’s campus at the beginning of the project and in the 
second year. FIPSE wanted the ad^ters to have a sense that they were involved in a larger project than 
the one at their own institution, and to see the program they were ad^rting in action on the mentor campus. 

Funding was also provided for the mentor to visit each of the adapting campuses once a 
year. Primarily intended to provide advice and encouragement, these visits also served to 
allow mentor interaction with a wide group of faculty and administrators on the adapting 
campus. In addition to providing opportunities to display the expertise of the mentors, the 
visits also allowed the mentor to run interference with administrators on behalf of the project, 
and for the adapters to publicize their efforts internally (and occasionally externally as well). 
All but one of the mentor-adapter clusters of institutions followed this model. The exception, 
the Stanford University-led service learning group, is discussed below. 

Each cluster was committed to using a common assessment strategy to measure the 
success of the project. In some cases (e.g., the Anne Arundel Community College and 
Moravian College clusters) adapters simply used the instruments that the mentor institution had 
employed. Others, such as the Stanford cluster, designed an entirely new assessment strategy. 
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The visits and the assessments provided numerous occasions for mentor and adapters to 
be in touch, and the mentors agreed to make themselves available by telephone or mail to 
answer questions as they arose, thus ensuring frequent, personal contact. However, contact 
among adapters between conferences was infrequent, although several clusters established 
automated Internet mailing lists or listservs for the purpose. 

Compared to FIPSE’s Comprehensive Program grants, those for “Disseminating 
Proven Reforms” were modest: $30,000 per year for the mentor and $10,000 per year for 
each adapter. Mentor funding was intended to cover costs for travel to the adapting 
institutions, the on-site costs of the conference, communication and materials costs and some 
release time. For the adapters, funding covered travel to the conference site for teams of three. 

The remaining funds were to be used at the adapter’s discretion in direct support of the program. 

Some Specific Examples 

The clusters described in the following vignettes are highlighted because of the overall 
success of their work and because they represent a range of the processes and outcomes that result 
from disseminating different types of innovations. Though these are singled out, others were equally 
successful. Indeed, no cluster was unsuccessful, and nearly all achieved what FIPSE had hoped. 

The range that these programs represent may be described by a simple typology that 
has proven useful for analyzing a variety of aspects of “Disseminating Proven Reforms.” The 
typology is based on the nature of the reform being disseminated, which affects the way that 
mentors and adapters interact, the nature of their cluster meetings and the mentors’ visits, the 
pace and manner of adaptation, and the assessment strategies. 

This rough but useful typology of reform “products” that are being “distributed” places 
them on a continuum of particularity, ranging from a fiilly defined package (Type 1) to a general 
concept (Type 4). At the more sp>ecific end are reforms like the California State University, Chico, 
introductory accounting sequence or Moravian College’s mathematics program, which consist of 
fully developed curriculum materials that adapters modified to suit their programs. By contrast, at 
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the general end one finds Eckerd College’s strategies for using retired professionals or 
Stanford s experience with community service programs, which served merely as prompts to 
inform the work of the adapting institutions. 

The mentors disseminating the more “packaged” reforms generally proceeded by 
instructing the adapters in the content and uses of the strategies and materials that they had 
developed. Thus the City College of New York (CCNY) “Fluency First” mentors conducted 
20-hour training sessions on each of their six adapting campuses. Similarly, the staff of the 
Anne Arundel Community College tutorial program devoted their opening workshop to a 
detailed exposition of their procedures and training materials. At the other end of the 
spectrum, Stanford’s mentors acted more as catalysts, arranging for institutions that were 
developing quite different community service models to share ideas with each other. 

To elaborate this description, we might place the 13 dissemination projects at four 
points on the continuum from particularity to generality of the product being disseminated, as 
follows: 

Type 1 . A set program: the disseminator had a fully developed package of materials 

to pass on to the adapters. Because the audiences of the adapters were the same as those of 

the mentors (e.g., students in an introductory math course or medical residents), the package 

could be taken over with only limited modifications. 

Anne Arundel Community College (“Supplemental Instruction”) 

California State University, Chico (introductory accounting) 

Moravian College (integration of precalculus topics into introductory 
calculus) 

University of Illinois College of Medicine (using patients as instructors for 
medical residents) 
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Type 2. A set strategy: the disseminator had a very specific way of going about the 

activities in which it was instructing the adapters, who then used that strategy to meet local needs. 

Adapters applied the strategy to audiences that were often different from those of the mentors. 

CCNY “Fluency First” (teaching reading and writing to ESL students) 
CCNY PASS Program (a school-to-college “bridge” program for students 
in science) 

Dickinson College (workshop physics) 

Pace University (case method in education courses) 

Type 3. A general strategy: the disseminator had developed an innovative strategy to 

meet certain goals which adapters modified freely to suit local conditions. 

Binghamton University (languages across the curriculum) 

Central Missouri State University (continuous quality improvement [CQI] 
strategies for academic programs) 

Eckerd College (senior professionals as student mentors) 

University of Missouri (teaching alternative dispute resolution in law 
schools) 

Type 4. A broad concept: the disseminator’s general approach to educational practice had 
wide latitude for applications, often in ways far removed from those of the mentor campus. 

Stanford University (service learning) 

As part of a continuum, these four discrete “positions” are not shaip divisions, and they 
tend to blend into each other. The projects highlighted in the following descriptions represent each 
of the four types. The Anne Arundel Community College project leads off not only because of its 
unifonn success but because it comes closest to following the pattern that FEPSE had in mind in 
laying out program guidelines. Moravian College’s “just in time” approach to teaching precalculus 
topics in the context of the introductory calculus course is the kind of tightly focused, 
department-specific innovation typical of Type I . The California State University, Chico, 
introductory accounting model (Type 1) illustrates how the institutional culture, particularly the 
faculty culture, into which the innovation is introduced can produce widely differing outcomes. 

The “Workshop Physics” group (Type 2), led by Dickinson College, demonstrates that 
institutions with quite different students and resources can all adapt a strong model with 
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demonstrably beneficial results. This group’s assessment process was especially strong. 
Another Type 2 project, City College of CUNY’s program for supporting at-risk students in the 
sciences, shows institutions adapting a basic idea to a wide range of administrative stnictures. 

The University of Missouri Law School and its adapters (Type 3) iUustrate the 
possibilities of using a formal dissemination process for introducing a paradigm-shifting 
change, alternative dispute resolution, into standard curriculum and teaching structures. The 
service learning cluster (Type 4), organized by Stanford University, exemplifies a successful 
approach to dissemination and assessment that is quite different from the rest. 

Lessons Learned 



The results of this first cycle of Disseminating Proven Reforms demonstrate the 
strength of the model. By a conservative estimate, 55 percent (39) of the 71 adapting 
institutions fully institutionalized the activity they set out to adapt, and another 25 percent (18) 
appeared likely to do so. For only five of the adapters did the innovation fail to take hold, for 
reasons ranging from outright rejection after a trial period to inaction. 

The effects produced by the model are not uniform across types. The more discrete 
and bounded the innovation, the better the model worked. Whereas 67 percent of the Type 1 
adapters institutionalized the innovation, only 57 percent of the Type 2s and 42 percent of the 
Type 3s have done so to date. Seventy percent of the adapting institutions where the innovation 
continues only in a weak form are of Type 3. (The single Type 4 group is omitted from this 
discussion because its dissemination approach was so different.) 

The Type 3 innovations, being less particular, broader and more complex, simply take 
longer to implement. CQI programs like Central Missouri’s, particularly when instituted in 
the context of academic programs, generally require far more than two years to take hold. 
Similarly, Eckerd’s successful efforts to bring retired professionals into the classroom in 
support roles required a substantial investment and administrative effort, not to mention faculty 
commitment, that may take a few years to put in place. 
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On the other hand, a mathematics department willing to try integrating precalculus 
topics into calculus courses using a “just in time” approach, rather than teaching the courses 
separately, has only to train itself to do so. No one outside the department, other than perhaps 
the campus curriculum committee, needs to be consulted. Thus the Type 1 and 2 innovations, 
which tend to be confined to particular departments and to require the consent of fewer people 
at lower administrative levels, seem most amenable to this kind of dissemination, at least in the 
short run. 

The Type 1 and 2 innovations, particularly the former, are less transfonnative, require 
less of a change in thinking and pose less of a challenge to the culture of the unit adapting 
them. Type 3 innovations, on the other hand, particularly those resembling the CQI and 
alternative dispute resolution programs, are truly revolutionary and thus harder to institute 
under any model. 

Much of the success of this model appears to lie in the skill of the mentors and the 
frequent contact of mentors and adapters. Apart from the personal qualities of the mentors, 
the regular schedule of meetings and visits provides incentive for the adapters to keep moving. 
The expert external to the institution is both a source of support and a reason to avoid the 
embarrassment of non-performance. 

Adapters rated personal contact with the mentor as most helpful. With few exceptions, 
telephone and e-mail contacts with the mentor were not initiated by the adapters, however, and 
tended to be used mostly for project logistics and administration. Most found the two group 
conferences useful as a source of mutual encouragement and a convenient means of conveying 
basic information about the details of an innovation and strategies for implementing it. 
Communication among the adapters outside of these meetings was virtually non-existent. 

FIPSE’s financial contribution, small though it was, appears to have been enough to 
generate the necessary effort. In fact, simply participating in a national project sponsored by a 
well-known funder is a powerful incentive. The formal recognition that selection implies 
moves a project to the top of the list of things to do. 
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Assessment tends to be the most difficult where the programs are the least defined, and 
where student outcomes lie beyond the grant period. Mentor institutions for most of the Type 
1 and 2 innovations adapted the assessment models of the original projects, but others had to 
devise new protocols, an effort that was particularly successful with the Dickinson, Stanford, 
and University of Illinois, Chicago Medical School clusters. Some clusters simply could not 
find a suitable process for project-wide assessment. 

The examples that follow represent all four project types and illustrate the successes 
and some of the disappointments that even strong programs must accept. 



SUCCESS OF ADAPTATION BY DISSEMINATION TYPE 



Type 

(# of Adapters) 


Fully 

Institutionalized 


Strong 

Continuance 


Weak 

Continuance 


Discon- 

tinued 


I (21) 


14 (67%) 


4 (19) 


0 


3 (14) 


n (21) 


12 (57%) 


7 (33) 


2 (10) 


0 


m (26) 


11 (42%) 


6 (23) 


7 (27) 


2 (8) 


IV (3) 


2 (67%) 


0 


1 (33) 


0 
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ANNE ARUNDEL COMMUNITY COLLEGE CLUSTER 
Supplemental Instruction with Mentoring Support 



PRINCIPAL MENTOR: Rosemary Wolfe, Education Program 

ADAPTERS: Ashland Community College, Community College of Philadelphia, Daytona 
Beach Community College, Dutchess Community College 



The Innovation 



Supplemental Instruction with Mentoring Support (SI) provides academic help to 
students enrolled in difficult required introductory courses (defined as those which students 
frequently fail or leave), while creating opportunities for faculty professional development. 

The program trains students and faculty to work together through a learner-centered approach 
of peer-group study and faculty mentoring support. Study sessions are led by community 
college students who have successfully completed the course and who attend classes again with 
enrolled students. During the study session, the student leader uses course material to teach 
skills for learning and organizing the content. Faculty, who are trained in learning strategies, 
attend classes and study sessions in courses outside their disciplines and serve as mentors to 
student leaders. 

Outcomes 



All four campuses established programs within the first year of the project, expanded 
them to additional courses, and plan to continue their work. Two of the campuses deviated 
from the mentor’s program by focusing part of their efforts on non-credit developmental 
courses, while a third targets bridge courses for English as a Second Language (ESL) students. 
Two of the adapters use the model solely in mathematics and science courses. One uses the 
instnictors who teach the courses, rather than faculty in other departments, to mentor the 
student leaders. 
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Two of the adapters were able to integrate program support into their budgets, because 
their units can designate funds for their own purposes. The others succeeded in getting long 
term allocations from higher levels in their institutions. The adaptations are fully 
institutionalized at all four colleges. 

This project followed the model plan of Disseminating Proven Reforms more closely 
than any other. The mentors conducted a workshop on their own campus, followed by site 
visits to each of the adapters in both years of the project, and a concluding workshop. The 
opening workshop for this group was exemplary. The mentor had prepared a large collection 
of materials: descriptions of various aspects of the program, forms for program administration 
and evaluation, reports of outcomes at the mentor institution, and hands-on exercises. 
Activities for the group were varied: short presentations, observation of various aspects of the 
program, conversations with student discussion leaders and faculty mentors, active learning 
exercises and discussions, individual team planning and meetings with the mentors. Timing, 
design of materials and practical arrangements had been carefully thought through. 

The initial site visits were used to provide an e^q)lanation of the program to a broad campus 
audience and to help the team with a range of practical details. The mentor was of particular help to 
some faculty project leaders who had little experience with administrative details and structures. 

Assessment 



The adapting institutions replicated the assessment that Anne Arundel Community 
College had used for its own project. The central question of this assessment was whether 
those students who chose to participate in the entirely voluntary supplemental instruction 
sessions earned a greater percentage of A, B and C grades (as opposed to Ds, Es and 
withdrawals) than those who did not participate. 

Across institutions, SI participants on the average earned significantly higher course 
grades and a lower percentage of D/F/W grades than nonparticipants. This result held for 
business, math, and science courses, which are the ones usually designated for SI support, as weU 
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as for ESL students. SI participants in developmental courses had final grades significantly 
higher on average than nonparticipants, but they did not receive a higher percentage of A, B 
and C grades. Developmental students also tended to attend both SI sessions and regular 
classes less assiduously. 

The superior performance of the SI participants did not appear related to any 
measurable academic superiority that they brought to the courses, although they may have been 
more motivated than nonparticipants. 

Surveys and observations of faculty teaching and faculty journals showed positive 
effects of SI on teaching. Faculty incorporated active classroom assessment techniques as a 
result of the program’s training and their participation as mentors. The interdisciplinary 
experience of having faculty become students in a content area outside their own promoted 
collegiality as well as a clearer understanding of how students learn. 

Further Dissemination 



Ashland Community College, through the Kentucky community college system office, 
and Dutchess Community College, through the New York State system office, plan to bring 
their work to the attention of institutions statewide. Daytona Beach Community College has 
begun to work with branch campuses to introduce SI. 

Available Information 



Further information may be obtained from: 

Rosemary Wolfe 

Fund for the Improvement of Postsecondary Education 
Washington, DC 20202 
Telephone: 202-708-5750 
e-mail: rosemary_wolfe@ed.gov 
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CITY COLLEGE OF NEW YORK (CUNY) ‘‘PASS” CLUSTER 
A Summer Bridge Program for Underprepared Students in Science 



PRINCIPAL MENTORS: Millicent Roth and Michael Weiner, Office of the Dean of Science 

ADAPTERS: Manhattanville College, Onondaga Community College, SUNY College at 
Brockport, SUNY College at New Paltz, University at Albany, University at 
Stony Brook 

The Innovation 



The Program of Access to Science Study (PASS) was designed by faculty at the City 
College of New York for underprepared students planning to pursue study in science-related 
fields and is offered in the summer before full matriculation. PASS consists of a preparatory 
science course with a special counseling component. This combination addresses 
underprepared students’ lack of problem-solving skills and their failure to become integrated 
into the academic and social life of the college. 

The adapting institutions were interested in the general principles of the City College 
model, modifying the form to fit local circumstances. Some stayed with the summer program 
model; some integrated similar efforts into academic-year programs; and some combined summer and 
academic-year activities. However, all retained the combined problem-solving and counseling focus. 

Outcomes 



In general, all the adapters were able to create a “bridge” course dealing with science 
topics and an accompanying counseling component. Most of the programs were able to 
identify sources of funding, either internal or external, which will support their continuance in 
fonns compatible with their original intentions, and reflecting the essential elements of the 
City College program. Furthermore, in the majority of cases, the PASS model was extended 
beyond the course or department where it was originally adapted into other areas. 
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For example, at one institution the project evolved from a summer preparatory program 
to an academic year program in the form of a “discussion group” offered in conjunction with 
the Biology I course. It includes a review of class material along with a PASS-style seminar. 
The discussion groups are also being offered with Biology II and with courses in general 
chemistry and organic chemistry. And they are open to all students, not just those identified as 
underprepared. At another campus, the initial summer program to prepare students for 
first-year college biology was expanded to support well-prepared as well as at-risk students in 
several science courses. At a third institution, the original adaptation involved a pr^>aratoiy course in 
the physical sciences for engineering and physical science majors. It has since been expanded to a 
preparatory course in the biological sciences for majors in that field. Students taking one of these 
PASS-style courses may use the course credits toward the fulfillment of degree requirements. 

These examples indicate the potential of the PASS model to take hold at adapting 
campuses. Continuation requires cooperation between academic departments and student 
support services, and shared enthusiasm for the program. 

Assessment 



The project directors employed an evaluator to track and assess student outcomes at the 
adapting institutions. This assessment, which looks at student performance and behavioral and 
attitudinal changes, has been completed for spring, summer and fall terms of 1996 and spring of 1997. 

During this period, the six institutions served a total of 157 students: 48 at Albany, 31 
at Stony Brook, 28 at Onondaga Community College and smaller numbers at the rest. The 
students were somewhat older than traditional entering students, markedly older at Brockport and 
Onondaga. About 75 percent were minority students. 

PASS students at four of the institutions had a higher retention rate after two college 
semesters than did the general college population. At one the rate was the same, and at 
another, marginally lower. PASS grades proved a moderate to strong predictor of grade point 
average, both overall and in math and science. Positive changes in student behavior during the 



PASS program also correlated strongly with future academic perfonnance. Given that 
underprepared students are less likely to persist and do well than the general college 
population, even outcomes for PASS participants similar to or slightly lower than those for the 
general population should be considered indicators of success. 

The best of these results closely track those achieved by the mentor institution. The 
assessment also provides data on such matters as student attitudes toward the program, which 
are generally highly favorable, and correlations between student self-assessments of behavior 
and faculty and counselor assessments. 

Available Information 



For further infonnation, contact: 

Millicent Roth 

Michael Weiner 

The City College of New York 

138th St. at Convent Ave. 

New York, NY 10031 
Telephone: 212-650-6768 / 8337 
e-mail: mikeweiner@aol.com 
e-mail: MTRCC@CUNYVM.CUNY.edu 
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DICKINSON COLLEGE CLUSTER 
Workshop Physics 



PRINCIPAL MENTOR: Priscilla Laws, Department of Physics and Astronomy 

ADAPTERS: Carroll College, Forsyth Technical Community College, Nebraska Wesleyan 
University, Skidmore College, University of Northern Iowa, Whittier College 

The Innovation 

Workshop Physics replaces traditional lectures and laboratories with guided inquiry 
workshops featuring microcomputers and specially designed apparatus to help students to learn 
by doing. Dickinson College pioneered this work in 1986 and has created an extensive activity 
guide and designed much of the software and apparatus to develop students’ reasoning skills 
and enhance their understanding of essential physics concepts. 



All six of the original cluster members continue to make progress in adapting 
Workshop Physics, albeit with varying degrees of success. The institution that progressed 
furthest began to implement Workshop Physics as soon as the project started in 1995, and used 
substantial portions of the workshop guide and pedagogical approach of the Dickinson 
program. Another, with extensive experience in activity-based instruction, used project 
participation to extend those instructional strategies to an additional course, adapting the spirit 
though not the materials of the Dickinson approach. 

At the other extreme. Just after the project began another adapting institution’s physics program 
was reduced to one full-time faculty member and assumed a purely service role. These changes severely 
limited the time available for implementing the extensive transformations that Workshop Physics 
requires, with the result that the course was not offered until fall of 1997. At another college, also with a 
declining science program, interest in the reforms has so far been limited to the project participants. 



Outcomes 
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The physics department at Forsyth Technical Community College found strong support 
from administrators and from faculty in programs that require physics. Another institution has 
found Workshop Physics particularly useful for training future science teachers. 

Four of the six institutions adapted large portions of the mentor’s materials and 
pedagogy. The other two, while including some workshop elements in their introductory 
courses, reinvented much material and strategy to better fit perceived local needs. 

All experienced common startup problems caused by demands on faculty time and the 
cost of and lack of technical support for an equipment-intensive program. Faculty were 
generally inventive in creating less expensive adaptations of the Dickinson laboratory exercises 
until they could acquire the necessary hardware. 

Some faculty not directly involved in the project resisted the workshop approach 
because of the loss of control that comes with the self discovery pedagogy, with the result that 
two of the institutions returned to substantial amounts of lecturing, while retaining some 
workshop elements. In doing so they were also motivated by the need to cover more material 
than discovery methods permit. 

Workshop Physics also encountered resistance from students who were not ready for an 
active learning situation and preferred to receive information through lectures. Some adapters 
found students unable to cope with the quantity of homework and the pace and scope of activities. 

Despite the large differences in the pace and degree of adaptation and in faculty and 
student acceptance of the new teaching and learning style, all six institutions introduced at least 
some elements of workshop physics. They believe their newly designed courses to be 
permanent and are convinced that students learn better than with the traditional approach (see 
evaluation results below). The variety of uses to which these institutions put Workshop 
Physics principles and the range of both type and extent of adaptation demonstrate the 
flexibility of this model. Thus the Workshop Physics project, now in its 14^ year, continues 
to change the way physics is taught across the country. 

The Workshop Physics program maintains an automated Internet mailing list or listserv 
that currently has 70 subscribers, a useful addition to person-to-person mentoring. 
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Assessment 



The University of Maryland’s Physics Education Research Group conducted an 
independent evaluation of learning outcomes at the adapting departments. The evaluation 
explored student gains in understanding of basic concepts characteristically covered in 
introductory physics courses and student attitudes, beliefs and assumptions about physics. 
Results for the Workshop Physics institutions were compared with results for institutions using 
traditional lecture methods and a mixture of lectures and hands-on methods. 

The gains in understanding of basic physics concepts were assessed by two instruments 
that in different ways measure students’ belief in the Newtonian laws of motion as opposed to 
their common sense beliefs. The two instruments, administered at the beginning and at the 
end of the course, were used at separate sets of institutions. Both showed students in 
Workshop Physics courses making twice or more the gains in understanding of students taught 
by lecture methods and 18 percent greater gains than those taught by mixed methods. 

The Maryland Physics Expectation Survey, also administered before and after the 
course, measures the difference between students’ expectations of and their actual experience 
in the course. It examines student beliefs about what it means to learn physics, how physics 
knowledge is structured, the nature of physics knowledge, beliefs about the connection between 
physics and reality, the role of mathematics in physics, and the effort necessary to do well. 

All courses showed poorer posttest results on the effort cluster of questions, possibly 
because students may not have put in as much effort as they expected to at the beginning. For 
the other five clusters of questions, the traditional and mixed sequences all showed 
deterioration of the pre-course results for at least one of the clusters, and four of the six 
traditional courses showed deterioration for at least three clusters. On the other hand, only 
one of the Workshop Physics courses showed deterioration on any of the five clusters. For 
three of the five clusters, all the workshop courses showed better outcomes than any of the 
lecture and mixed-method courses. 
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The evaluators also conducted interviews with 16 individual students at two of the 
adapting institutions in order to get more detail about the thinking that underlay responses to 
the expectations instrument. They found that students clearly saw connections between 
classroom learning and phenomena they encountered outside the classroom, and that most were 
convinced that they really could learn physics. Though most felt that they needed to 
understand the concepts in order to learn, about a third still felt that they could pass the course 
by memorizing equations and plugging them into problems. Nevertheless, most of the students 
would rather have been in lecture courses where they were told the concepts rather than having 
to figure them out for themselves. Much of this preference stemmed from their uncertainty 
that what they were learning on their own was “right.” 

The students also had a mixed, but largely favorable, reaction to working in groups. 
Although personal relationships were occasionally a problem, more often difficulties arose 
between academically weaker students, who were interested in getting through the work as 
quickly as possible, and their stronger peers, who wanted to make sure they understood the 
concepts before moving on. 

Further Dissemination 



Workshop Physics has been one of the most extensively disseminated innovations in 
higher education. A series of grants from FIPSE, the National Science Foundation and other 
sources have supported many well-attended workshops and conference presentations. This 
project, however, is the first in which the developers of the program have been able to work 
with institutions over an extended period of time. Conference presentations continue, as does 
dissemination through a Workshop Physics Web site. 
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Available information: 



For further information, contact: 

Priscilla Laws 

Department of Physics and Astronomy 

Dickinson College 

P.O. Box 1773 

Carlisle, PA 17013 

Telephone: 717-245-1242 

e-mail: lawsp@dickinson.edu 

Web site: http://physics.dickinson.edu 
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UNIVERSITY OF MISSOURI SCHOOL OF LAW 
Integrating Dispute Resolution into Law School Courses 



PRINCIPAL MENTOR: Leonard L. Riskin, School of Law 

ADAPTERS: DePaul University, Hamline University, Inter American University of 
Puerto Rico, Ohio State University , Tulane University, University of 
Washington 



The Innovation 



Traditional litigation, though sometimes the best way to settle differences, has given 
rise to a tide of dissatisfaction. Complaints include high cost, delay, emotional trauma for the 
parties, and inadequate remedies. Each of these deficiencies steins in part from the tendency 
of law school education to focus on litigation and the adversarial view of human relations on 
which it is based. In response to the problems surrounding traditional litigation, an array of 
programs has been developed to foster alternative methods of dispute resolution, commonly 
called ADR, for alternative dispute resolution. ADR includes such processes as negotiation, 
mediation, and arbitration or combinations thereof. 

Beginning in 1985, the University of Missouri, Columbia, School of Law 
systematically integrated alternative dispute resolution, taught through simulations, into all 
standard first year law school courses. That project produced law school course books, an 
instructor’s manual, and a series of videotapes of simulated situations. Through this 
dissemination project the university’s Center for the Study of Dispute Resolution supported six 
institutions in creating and strengthening ADR instruction for law school students. 

Outcomes 



All six adapting institutions introduced or expanded instruction in alternative dispute 
resolution in ways that appear to be permanent and promise further expansion. Between one and 
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seven courses, many of them first-year, were substantially altered to incorporate material on 
ADR. All schools added specialized dispute resolution courses at the upper level in the form 
of either classroom or clinical course offerings. 

The disseminators’ initial expectations that the adapters would follow their lead in 
integrating ADR teaching into first-year law courses early proved impossible to achieve. The 
project director posits the need for four conditions in order for this full-scale integration to 
take place: 1) at least one knowledgeable and committed lead faculty member who has time to 
work with colleagues; 2) a core of at least three other like-minded faculty members; 3) strong 
and visible support from the law school dean; 4) a consensus of the majority of the faculty, 
even if weak, that ADR should have a place in the first-year curriculum. The first condition 
held at all six institutions and the third at the four institutions that were not in the process of 
searching for a dean. The second and fourth conditions prevailed only at those two schools, 
Hamline and Ohio State, that had worked with ADR for several years in clinical settings and 
in specialized advanced courses. 

The other four institutions chose to involve only those faculty members who were 
interested, and to place the ADR activities where such interest created opportunity. All made 
some attempt to include ADR in first-year courses, with outcomes ranging from substantial 
attention in one or more key courses to ADR simulations in large enrollment courses. The 
first-year legal writing course, particularly because it is taught in small sections, seemed to be 
especially hospitable to ADR materials and exercises, but ADR got attention in a wide range 
of courses taught by interested faculty. 

Resistance to ADR had many sources. Some faculty simply did not believe in it or 
thought that it did not have a theoretical base. This was an attitude of which many were 
disabused through workshops and clinics conducted in the course of this project. Others 
worried about losing coverage of other material if ADR activities were introduced. 

Discomfort with the simulation process, especially the loss of classroom control, was a 
prevalent source of resistance. Still others saw difficulty in introducing hands-on activities 
into large courses. 
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. Project leaders at the adapting campuses dealt with these problems in ways ranging 
from personal persuasion to workshops to semester-long training sessions enrolling both 
faculty and advanced students. Others made limited efforts at conversion and worked around 
the resisters, and one largely proceeded on his own. 

At the end of the project’s two years, two institutions (Inter American University and 
DePaul) had established negotiation and mediation institutes, in addition to Haniline, which 
already had one. Still another university was seeking funding for such an institute. Ohio 
State’s College of Law had been given funds to add two new faculty positions in dispute 
resolution, in light of a general faculty decision to make dispute resolution a major part of the 
instructional program. The University of Washington and DePaul had created curriculum 
tracks in dispute resolution. 

This project gave rise to a substantial amount of material for publication. Much of it is 
in the form of exercises and teaching materials that appear in Instructor's Manual to 
Accompany Dispute Resolution and Lawyers, by Leonard Riskin, James Westbrook and James 
Levin, 2nd ed. (West Publishing Co., 1998). 

Particularly useful were two guides to briefing cases in a way that identifies the 
underlying interests of the parties to the dispute, rather than simply the applicable points of 
law. These guides were produced independently by two faculty members at two of the 
adapting institutions as a result of discussions at the second of the project’s group conferences. 

The principal mentor was unusually effective in offering workshops and demonstration 
classes which, among other things, served to persuade some faculty that ADR does indeed rest on a 
strong theoretical base. Particularly eneigizing for many adapting-team members was the prospect of 
publishing teaching materials and simulation exercises through the mentor institution’s contracted books. 

This project has been recognized through an award to the principal mentor and the 
University of Missouri-Columbia School of Law from the CPR Institute for Dispute 
Resolution, a coalition of leading corporations and law firms with offices in New York City. 

The award was for “Pioneering Work Introducing ADR into the Curriculum at the University of 
Missouri-Columbia School of Law and Assisting Other Law Schools in Developing Similar Programs. ” 
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Assessment 



Hamline undertook an evaluation to see how students’ attitudes toward ADR had 
changed as a result of instruction in the first-year course. Overall, they found an increase in 
students’ orientation to ADR, largely as a result of the changed attitudes of women students. 
Women began the course with a more negative attitude toward ADR than men but moved 
strongly toward ADR by the end of the course. Men, on the other hand, actually moved 
slightly in the direction of adversarial proceedings. 

Overall project evaluation was performed by an external evaluator who had served in 
that role for the University of Missouri’s two earlier FIPSE-funded projects. Because the 
adapters were at such different stages of introducing ADR and took such different approaches 
to their work, no common statistical evaluation was possible. Thus the evaluation consists of a 
thoughtful essay on the nature of the innovation and its possibilities for dissemination. 

The evaluator notes that ADR appears to most faculty as heretical, challenging 
prevailing legal practice and, in its emphasis on active simulation, the prevailing pedagogy of 
law schools. For dissemination to be successful, these heresies must in some way be “tamed,” 
presented in such a way that they are at least tolerable to a majority of the faculty. 

The taming procedure inevitably results in a number of substantial changes, such as 
putting the pure dispute resolution strategies into a context of pre-trial negotiations in which 
the resolution occurs under the threat of eventual litigation. Although this solution is 
comforting to traditional faculty, it misses the point of ADR. 

Compromises of preferred instructional procedures may also be necessary. If ADR is 
integrated across many courses, it must depend in part on marginally sympathetic faculty who 
do not understand how the simulations work and are not entirely comfortable with them. In 
both substantive and procedural cases, the danger lies in compromising basic intentions beyond 
recognition. Apparently this has not happened at either the mentor or the adapting institutions, 
probably because implementation was conducted slowly and carefully and left in the hands of 
committed faculty until others could be trained. 
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In view of the nature of the innovation and the difficulties of disseminating it is 
remarkable that the adapters succeeded in establishing a firm presence for a radically different 
approach to the training of lawyers. 

Further Dissemination 



The original University of Missouri project produced a steady stream of publications. 
The major products of the dissemination project itself are the teaching materials and 
simulations included in the instructor’s manual for the second edition of Dispute Resolution 
and Lawyers, mentioned above. A 1998 symposium issue of the Florida Law Review focused 
on the dissemination project and included articles by the principal mentor, the evaluator, and 
five professors from adapting institutions. 

Available information: 



For further information, contact: 

Leonard L. Riskin 

Center for the Study of Dispute Resolution 
School of Law 

University of Missouri Columbia 
Columbia, MO 65211 
Telephone: 573-882-8084 
e-mail: riskinL@missouri.edu 
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MORAVIAN COLLEGE CLUSTER 
Integration of Precalculus Topics into the First Course in Calculus 



PRINCIPAL MENTOR: Doris Schattschneider, Mathematics Department 

ADAPTERS: DePauw University, The George Washington University, Hudson Valley 
Community College, University of Puerto Rico-Mayagiiez, University of 
Puerto Rico-Rio Piedras, United States Military Academy 



The Innovation 



Students who come to college unprepared to enroll in a required calculus course 
characteristically take precalculus first. There they study a number of topics they will need in 
calculus, but often they study them with no sense of the relationship of those topics to their 
later study in mathematics. By the time the students need to apply their prior learning in the 
calculus sequence, they have often forgotten what they learned earlier. 

To address this problem, Moravian College developed a two-course sequence, 
“Calculus I with Review,” integrating precalculus and the first semester of calculus. 
Precalculus topics are introduced at the time they are needed to understand the calculus 
concepts. This “just in time” approach is supported by a published text, A Companion to 
Calculus, developed by members of the Moravian College faculty. Adapters used this text 
along with their regular calculus in their newly established integrated sequence. 

Outcomes 



At four of the six adapting institutions the new integrated course is fully 
institutionalized. Its status is somewhat uncertain at the two University of Puerto Rico 
campuses, where the mathematics department has somewhat less control over course placement 
and course approval processes are more cumbersome. 
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This innovation apparently responds to widespread frustrations of mathematics faculty 
about the precalculus-calculus sequence. Though each campus experienced resistance to the 
new course from some faculty members, the superior effects on student persistence in the 
integrated sequence were hard to argue with. And where students had a choice of pursuing an 
integrated sequence rather than the traditional sequence they overwhelmingly chose the fonner. 
In view of the evidence, dubious faculty had little choice but to accede to departmental 
decisions to adopt the integrated sequence exclusively. 

Of all the innovations disseminated through the “Disseminating Proven Reforms” 
competition, this one seems the most readily and broadly transferable. It deals with courses 
that virtually all institutions offer on a large scale and addresses a widely felt need. It is 
accompanied by strong, straightforward assessment questions and methods that seem always to 
show better student retention for the integrated course than for traditional courses. Finally, 
although it requires faculty to change the logic and organization of their courses, it does not 
require them to change their teaching styles. 

Assessment 



The adapters used the same assessment strategy that the mentors had when they 
instituted the original program. They looked at: student persistence rates from the first 
semester of the sequence to the second, and rates of completion for the second course; the 
performance of integrated-course students compared to students in the traditional sequence on a 
set of problems included in the final examinations of both courses; instructor attitudes, 
including comparative assessment of the integrated and traditional sequences; and student 
attitudes. 

Uniformly, student persistence through the sequence was higher for the integrated 
course than for calculus preceded by precalculus. Integrated-sequence students performed at 
least as well on a set of common problems as the traditional-course students, and sometimes 
better. In general, both faculty and students liked the integrated sequence better. 
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Further Dissemination 



Moravian faculty continue to get inquiries about their course and to be invited to make 
presentations. During the time of the project they and the adapters made eight conference 
presentations. As a result of the work at Hudson Valley Community College, the Moravian 
group has been invited to work with faculty at the State University of New York, Albany, 
which is the main transfer institution for Hudson Valley students. The project Web site is 
located at the community college. 

Available information: 

Further information may be obtained from: 



Doris Schattschneider 
Mathematics Department 
Moravian College 
1200 Main St. 

Bethlehem, PA 18018 6650 
Telephone: 610-861-1373 
Fax: 610-861-4162 
e-mail: schattdo@moravian.edu 
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STANFORD UNIVERSITY 
Building an Extended Community of Service Learners 



PRINCIPAL MENTOR ; Timothy Stanton, Haas Center for Public Service 

ADAPTERS ; California State University-Monterey Bay, Mills College, Portland State 
University 

The Innovation 



Since the founding of Campus Compact more than 15 years ago, service learning has 
become a familiar feature of undergraduate education. Stanford, whose former president 
Donald Kennedy was one of Campus Compact’s three founders, has been a leader in this 
work, with many of the university’s programs emanating from the endowed Haas Center for 
Public Service. 

Service learning is distinguished from other types of experiential learning by its special 
concern that students develop habits of community service as part of their undergraduate 
education. Unlike internships or cooperative education, service learning does not seek to 
contribute to students’ career development (although it may) but rather to draw students’ 
attention to the nature and obligations of living in a democratic society made up of many 
different kinds of people. Service learning proceeds from volunteerism but puts that impulse 
within specific intellectual, often academic contexts. 

While student affairs offices have been the locus of volunteer activities and are the 
organizers of much service learning activity, academic, credit-granting programs have 
increasingly adopted service learning as a required aspect of courses or as the focus of whole 
field-based learning experiences. Stanford has been successful in finding ways to incorporate 
service learning into the intellectual traditions of the university in a wide range of departments, 
including opportunities to use service learning experiences as the basis for student research. 
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Outcomes 



Sewice learning differs from other innovations in “Disseminating Proven Reforms” in that it is 
not a defined and specific program. Rather, it encompasses a wide range of activities and takes many 
forms, depending on institutional context. Unlike, say. Supplemental Instruction (Anne Arundel) or 
Workshop Physics (Dickinson), service learning is not a single structure and set of procedures but rather 
a commitment to particular principles and emphases in undergraduate education, accompanied by a range 
of pedagogies that are still developing and e?^)anding. Therefore, the participants in this cluster did 
not expect to take specific activities that are successful at Stanford and adapt them but rather to 
use Stanford’s longer experience and pedagogical principles to guide their diverse programs. 
The group adopted a “co-mentoring” model of dissemination whereby each of the participants 
brought its particular experiences to the table for the benefit of the others. 

Rather than Stanford mentors hosting conferences and visiting each of the adapters in 
turn, the group met quarterly during the two years of the project, rotating the meetings among 
the partner campuses. The host campus was responsible for organizing a two-day program on 
a mutually agreed theme that evolved into a “curriculum” for the participants. Topics for the 
meetings included assessment, embedding service learning in the curriculum, community 
partnerships, faculty scholarship, and institutional culture. 

At project’s end, the four campuses that made up this cluster were in quite different 
situations with regard to service learning. Stanford, with the most mature program, continued 
efforts to develop new programs and to extend existing ones into new departments, thus 
increasing both faculty and student participation. California State University, Monterey Bay, 
created two years earlier with service learning as a defining feature of its mission, was in the 
process of determining just how that mission would translate into curriculum and pedagogy. 
Mills College, a small liberal arts institution, was just beginning a serious effort at service 
learning. During the project period, loss of internal funding for service learning and instability in 
program leadership substantially retarded efforts. A recent large grant to expand the multicultural 
aspects of the curriculum includes creation of 40 new service learning courses. 
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Portland State recently reorganized itself and its curriculum to focus on its relationship to its 
urban community. Service learning is a requirement for sophomores and, as a general 
education capstone experience, for seniors as well. 

In summary, the two independent institutions are seeking to expand their activities 
within an established structure, one with stable leadership and substantial funding, the other 
with fluctuating funding and changing program administration. Tlie two public institutions, 
one newly created and the other significantly transfonned, have made service learning a 
defining focus. 

The participants felt that individually and collectively they had made progress in the 
following areas: deepening practice and developing the field, creating a professional 
community, further validating and legitimizing service learning on campus, encouraging and 
strengthening continuous assessment of service learning, developing a renewed focus on and an 
understanding of students engaged in service learning, recognizing the importance of 
institutional context in the process of institutionalizing service learning, and developing new 
strategies for research and assessment. 

It is difficult to cite concrete examples of these accomplishments, although many of 
them were visible in the context of the rich conversations at the quarterly meetings. For the 
most part, the outcomes are expanded knowledge and awareness resulting in improved 
program content and process at the member institutions. The group members, all leading 
administrators of service learning programs at their institutions, provided samples of the direct 
experience of service learning participants and some unusually sophisticated reflections on the 
programs. This is a model of mutual mentoring through intense, structured conversation 
carried on for an extended period of time. 
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Assessment 



Each of the institutions assessed their progress in terms of a set of “indicators” 
developed throughout the project. Some of these indicators, such as assessment efforts, 
program sustainability, and embedded service learning, were used by all the campuses. Some 
individual participants added their own, such as formation of community partnerships and 
faculty scholarship. In the final report, each campus listed specific examples of progress for 
each of the indicators. The structure of the partnership as a whole and specific areas of 
progress were examined, described and commented on by an external evaluator, who 
attempted to estimate the project’s effects on the partners. The evaluation emphasized 
strategies and improved practice as opposed to numbers of new courses, because three of the 
four institutions already were carrying out large service learning programs 

The participants also developed a joint research project in each institution and 
interviewed students who had participated in various kinds of service learning activity. From 
these interviews they are developing portraits of typical students to serve as benchmarks in 
assessing the effects of their programs. 

Available Information 



For further information, contact: 

Timothy Stanton 

Haas Center for Public Service 

Stanford University 

Stanford, CA 94305-8620 

Telephone: 415-723-2859 

e-mail: cr. tks@forsy the. Stanford. edu 
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